Friday, December 25, 2009

Freerunner Battery Mod, Case Mod, Runs Android

Oklahoma was hit by the biggest snow storm in state history this Christmas. Since we were snowed in and couldn't visit relatives, I found myself with some unexpected time on my hands. So I built my own Android phone.


I bought a Neo Freerunner last year, but some design flaws kept it from being usable to me as an everyday phone: pitiful battery life, audio problems, and no single Linux distribution stood out as the one best choice to use. But the battery was the worst problem.


My previous experiments had shown that if you remove the battery monitor circuit board from the original battery:


...you can transfer it to a larger Li-Ion battery, like this one from a portable DVD player, and it will work. Surprisingly the BQ27000 chip on the smart battery board is able to learn the larger battery capacity.


Of course, I didn't let the old battery go to waste either...


The prototype battery mod worked but I needed to give the phone a bigger enclosure so the battery wouldn't have to be on the outside where it might be damaged.


Having proved the concept, I exchanged the junk DVD player battery for a brand new 6 AH Li-Ion battery from Sparkfun, and put it all into the case from a 2.5 inch portable hard drive. (To put that in perspective, the original battery was only 1100 mAH.)


Rather than try to cut the hard drive case to perfectly match the Freerunner circuit board, I simply reused the original phone faceplate since it already had the proper board mounting locations. I cut an oval hole in the hard drive case (free hand with a dremel), and then used JB Weld to join the two pieces and to even out the gaps between them. I put the JB Weld on sloppy but used a wet paper towel to wash away the excess, leaving a nice even seam all the way around. (Turns out JB Weld is water soluble - who knew!)


By the way that silver squiggle shape on the front is the cellular antenna. I ran out of room on the inside, so I put the antenna on the outside! And putting the antenna next to my cheek is a good way to extend a sarcastic "bite me" to all those idiots who argue from a position of ignorance that cell phone signals cause cancer without knowing the first thing about the physics of radio signals.


Although from the faceplate it may appear as though an entire Freerunner has been simply embedded into the new case, behind the faceplate there is nothing left of the original Freerunner case, and some components have been moved and some improvements made as I tried to rectify my main complaints about the original phone.



Here it is charging at a total rate of nearly 2 amps using both the fast charge board and the Freerunner's internal charging circuitry. No apparent harm comes of running them both at once, and the extra charger doesn't confuse the battery gas gauge chip either.


The BQ27000 learns the battery capacity by observing a complete battery discharge cycle. Unfortunately software will shut down the phone when it thinks the battery is empty, so we'll never actually reach a full discharge of the new battery to recalibrate the gas gauge! Luckily I built a battery discharge device a while back, which is really just a mess of enormous low-ohm power resistors strapped to a massive heatsink with a fan on top. And a fancy digital temperature readout so I know whether I'm actually frying my power resistors. This discharges this battery at a rate of 1 amp, which is a bit too fast, but with a 6 AH battery I could be waiting all weekend for it to discharge at "normal" current draws.


The documentation for the BQ27000 gas gauge chip mentions that it doesn't revise its capacity estimate downward more than 1/8th of the total at a time. It doesn't say anything about how the estimate gets revised upward, but experimentation with the test battery from the DVD player indicates that it caps the upward change at 1/4th of the previous estimate each time. The test battery reached 2.2 AH, so I can only expect the BQ27000 to revise the estimate up by about 0.55 AH on this cycle. It's probably not worth the trouble to work it up to the full 6 AH reading, since the process takes 12 hours and I'll be slightly damaging the battery every time I completely discharge it! Maybe when the battery is deteriorated in a few years, I'll calibrate it to whatever reduced capacity is still in it.


Not really visible in the pictures, but I removed the speakerphone (which was useless in its old location because its proximity to the microphone made it impossible to actually use speakerphone without getting a loud feedback squeal) and connected it in place of the earpiece. I thought that, because it was bigger, it would be louder than the earpiece, but instead it is much softer, nearly inaudible. I think that's because the earpiece measures as high impedance while the speakerphone speaker is low impedance, so there isn't an efficient transfer of power. Tomorrow I'll cannibalize the audio amplifier IC from that same portable DVD player I got the test battery from, and use it to fabricate a tiny audio amplifier board to drive the earpiece & match the impedances. (I can design a circuit in my head and build it on a new board as fast as I can take the components off the board I'm cannibalizing.)


That will also provide me a physical volume control so I won't have to rely on the goofy software volume settings that never seem to work right on the Freerunner. Hmmm... Maybe I'll make it stereo while I'm at it. The Neo 1971 (prototype to the Freerunner) had stereo speakers in it, but not the Freerunner. The audio amplifier chip in the DVD player is stereo, and I can pull a stereo signal from the headphone jack. Interesting...


So... Android right? Remember at the top I said that my original goal was to address some specific flaws with the Freerunner, one of which being lack of a definite distribution to stick with. I didn't actually like Android when I first tried it (still not sure about it) but it seems to be where the ball is rolling. When I began to think I wanted a Motorola Droid, and when blogs and podcasts started going crazy over the (latest) rumored Google phone, I had to stop and remember, wait a minute, I already HAVE hardware capable of running Android. It's just been sitting on my desk in pieces waiting for me to put it together.


Installing Android was a breeze; easier than any other install I've done on the Freerunner. You just untar a file onto an SD card, and then booting off the SD card prepares Android on the Freerunner. No messing around with the flashing the phone over USB, and no messing around with bootstrapping the system on the phone itself like Debian does. Just pop in the SD card and it just works. Not much else to it - except wifi doesn't work. Why does wifi never work for me on Linux? Oh well...

Friday, October 9, 2009

Anything That's Not Tied Down

This morning when I got out of class I found someone had undone several of the velcro straps that hold my electric bicycle's battery on. Fortunately the battery (and the bicycle!) were still ok.

My guess is someone just saw a bulky black bag and decided to steal it without knowing what was inside. Walk-by snatch and grab. I like to think the sheer weight of 30 lbs of lead acid batteries scared them into letting go of it. And anyway there's more straps holding that thing down on the front side.

Saturday, October 3, 2009

Everything Takes Longer In C++

So I had tonsil surgery last Monday. Even though I didn't feel up to programming for my day job, I couldn't stay away from trying to work on the MMO game project called Emergence that Ben and I are creating. When it seems like progress is going glacially slow on our game Ben and I always say "if only we had more free time to work on this..." Well I've had time this week - and I still made only baby steps in progress on it. Granted I was resting from surgery and on pain medicine, but still... I'm beginning to think it's not (just) that we have no free time. It's that with programming you put in enormous effort yet make only incremental progress.

If it takes a certain amount of effort to write software of a given complexity, writing software that's twice as complex doesn't take twice as long, it takes, I don't know, ten times as long, twenty times as long. The effort increases exponentially. Invert that and you get an equation that says that you only get logarithmic increases in output for every increase in effort. If you don't know what a logarithmic curve looks like, it has a knee bend. If you stay to the left side of the curve you get pretty good output for input, but as you move right it levels off quickly. To the right side of the curve, increasingly large increases in input result in increasingly small changes in output.

So I think I've been working dangerously close to right side of the curve lately, where what you can accomplish tapers off faster than the increase in effort. I've been getting better with some of the new programming techniques I'm using, and I've streamlined my build process and improved my debugging techniques. But here's the funny thing - all those improvements really just help me increase my work input. They don't directly increase the output. You see if I streamline my build process and my debugging techniques so that I can make a change and test it in half the time, that just means I can test twice as many code updates in the same amount of time. But if I'm on the inefficient side of a logarithmic curve, I may actually need to make four, or ten, or twenty times as many code changes at a time to actually double my output.

So that's what's so frustrating about this work. You run faster and faster but never seem to get anywhere. It's not hopeless though. I think the fact that the ratio of accomplishments from effort is logarithmic is some kind of fundamental law of software, but it is possible to tweak the shape of the curve. Although it will always eventually taper off, it doesn't need to taper so as quickly. I think changing the programming language you program in does the most to change this curve. C++ is very powerful and generates efficient programs, but it has a very sharp bend in its complexity curve. This is because doing anything substantial in C++ just involves so many fiddly little details, and the language does so little for you automatically. I'm not saying you can't build large systems in C++, but I am saying it probably took exponentially more work to do it than it would have to do it in a scripting language. And I'm not saying you shouldn't use C++; in order to get good performance in our game, we need to. But we're going to use both C++ and script language in our game, and I need to get out of the C++ part and into the scripting as soon as possible.

The ironic thing is that the component which is giving me all this trouble, where I'm working too far on the right side of the complexity curve, is the C++ component that will bind our C++ side of our game to our scripting side. In order to leave C++ behind I need to first overcome a trial by fire of the toughest C++ I've ever worked on.

Friday, September 4, 2009

Electric Bicycle Conversion


A student parking pass at the University of Oklahoma costs ~$200 a semester. As if that weren't bad enough, they first try to trick you into buying the nine hundred dollar reserved pass. Instead of paying "the man" $200 just so I can park in overcrowded, confusingly marked lots, I decided my money would be better spent upgrading my bicycle so I can park a few miles away and bike to class.




I knew about these electric bicycle conversion kits because a couple of CS Ph.D. students I'm friends with, Lawrence Kinchloe and Mark Woehrer, put electric conversion kits on their bicycles a couple years ago. (Hey guys! We should start a club!)

It turns out WeRelectrified.com sells a low-end, but complete bicycle conversion kit for $200. (Ha! Exactly the same as I would have spent on a parking pass!) While I could have fabricated my own motor controller, and scrounged my own batteries, the complete kit is such a good deal that it didn't make sense not to just use it as-is. There's even more than what's in the picture - it also comes with a metal rack to go on the back of the bike for the batteries.




The way this thing works is there's actually a motor inside the front wheel, a big fat metal hub that makes the front wheel turn by itself. (Nothing gets changed on the back wheel - so you can still pedal it too, like a normal bike.) In theory, you can just replace the front wheel of your bicycle, put the batteries on the back and the throttle on your handlebars, and you're ready to go!




That's the idea, anyway. Of course, this is a Project, and as we all know Projects rarely go according to plan. The first problem I had was that the hub is too wide to fit between my front forks. Earl Martin would probably just have grabbed the front forks and pulled them out, but I'm not that strong. Instead, I took the scissor jack that came with my car, slipped it over the front forks, and just cranked it on out there. (Nick Johnson asks "Is that SAFE?" Come on, Nick, it's a steel frame! You could probably safely bend it into a pretzel if you wanted to.)




OK, now the hub fits - but the axle is too big to go in the notches on the end of the forks.




Not a problem for a man who owns more kinds of steel files than socks! This is a delicate operation - the axle has flat sides on it, and I want the notch just wide enough for the flat sides to fit, but not wide enough for the axle to rotate in place. When the hub motor turns the wheel, it's pushing against the axle to do so, and you don't want the axle to be able to start spinning within the forks.




Great! Front wheel fits now!




OK, now to transfer the tire from the old wheel to the new wheel. Except the tire doesn't fit. In fact, these aren't the same size at all! Did I buy the wrong size wheel? Oh, never mind. This is the same thing I went through last time I worked on this bicycle! The wheels I put on it then were narrow 26 inch, which means they have less tire material and more rim circumference than the moutain bike wheels it came with; the new wheel brings us back full circle, with smaller diameter rims under thicker tires. Means I need to buy a tire.




Off to the bicycle store!



Since the front wheel is now a different size, I decided to change the back wheel to that size too, and buy new tires for front and back. I've been meaning to replace the back wheel anyhow (between either the wheel starting out warped or my asymmetric weaving pattern, the wheel was getting ever so slightly out of round). I didn't think I needed to buy new gears though because I could just take the gears from the old wheel, right?




Oops... that's not how you remove the gear pack. (If ball bearings fall out all over the place, you're doin' it wrong.) If I read my own damn blog from last year I MIGHT have remembered that. I never was actually able to get the center portion of the hub off.




BACK to the bicycle store!




Another one hour drive. This time I came back with a gear pack. I put yellow teflon tape on the threads of the new back wheel - hopefully if I ever need to remove the new gear pack, this will keep the gear hub from getting stuck as badly as the old one did.




Then I remembered that on the second trip to the bicycle store, I forgot to return the inner tubes I bought on the first trip. The guy at the store mixed up the two sizes of 26 inch just like I did with the wheels, and sold me the skinnier, larger circumference narrow tubes when I needed the wide 26 inch tubes. These tubes don't fit in the tires I bought (there's excess tube), but by this point I'm beyond caring so I just jammed them in there anyway, kinks and all. I haven't gotten a flat yet so I guess it's OK.




I'm kind of proud of the way the throttle mount turned out. I didn't want to unwrap all my handlebar tape to slide the throttle assembly on, so I wasn't sure how I was going to mount it. Eventually I realized it would just as easily fit on a piece of 3/4 inch PVC pipe, so I mounted the throttle vertically on a piece of pipe. I knew if a convex pipe edge butted against a convex handlebar edge it would never stay in place, so I cut a half-circle indention in the side of the PVC pipe where it mates to the handlebar. The mounting feels solid and the throttle is not any harder to use in this position.




It has a key for an on/off switch. It happened to be a perfect fit for this space under the bicycle seat. While it might look a little rude if I reach between my legs to turn the key, I figure this will keep the rain off it.




I mounted the motor controller box upside down under the battery, and I made all the wire connectors come out in the space hidden between seat and the battery bag. Hopefully that will make the wiring less obvious from above. Although I'm not doing anything wrong, I don't exactly want to advertise that something strange is going on. My experience with authority has been that even harmless things will be treated with suspicion if they out of the ordinary. Besides, I don't want to give potential bicycle thieves any reason to think my bicycle is special.




Here's the completed bicycle. As long as I had to buy new tires, I thought I'd have a little fun and get ones with a red stripe. I think it's a nice effect. You can see the batteries here in the unzipped bag; it came with a block of styrofoam in the back where I assume the 3rd battery would go if this were a 36 volt kit instead of 24 volts.



On my first day riding it to class, the zipper failed on the bag where the corner of the battery meets it. The batteries, which are sealed lead acid (heavy as hell but cheaper than lithium ion), rest directly on the metal frame of the rack. There's no padding in the bag, so with every bump the batteries slam down against the rack and then up against the zipper.

Catastrophic zipper failure!




When this happened, I wasn't even sure how I would secure the batteries for the ride home. I actually have a pair of shoes that I've threaded two sets of laces into for MacGuyver emergencies. I could have taken out one set of laces to tie down the batteries and still have had the other set of laces to keep my shoes on. Wouldn't you know it but this would be the day I wore a different pair of shoes though? Darn the luck. Fortunately I had previously fixed the water bottle holder with baling wire, and there turned out to be enough when I unwrapped that to hold the batteries down.




The foam from this padded laptop bag will be just what I need to cushion the batteries. After I cut out the foam, I'll fold it in half and put it under the battery bag.




A zipper is completely off one side. How will I get back it on track? By cutting deeper into the throat of the zipper, I was able to give myself enough slack to re-thread it from the end.




Then I made the two zippers meet at the corner where the original tear happened, so I don't ever have to cross the damaged section! The amount of offset I created from rethreading the zipper on the left side happens to the same length as the stretched portion here, so everything matches up properly.




The next time I have something go wrong with the bicycle, I'll have some duct tape stowed onboard. I removed the hard cardboard center from a roll of duct tape so I could fold it up. I think I saw that trick on Instructables.




And there it is, back together and nicely padded.




Unfortunately the cheap made-in-china 24V battery charger that came with the kit died after just one use! I can use my car battery charger to recharge the batteries, but to do so I have to change the wiring on the battery pack from a 24V serial configuration to a 12V parallel configurations every time I hook up the 12V charger.




Finally done, fully charged and put back together. Despite the problems, I would say the project has gone better than I anticipated. It's lots of fun to ride and completely practical. I think if more people understood that we'd see as many electric bicycles in stores and on the road as regular ones. It still works as a bicycle, so you can rely on electrical power as much or as little as you want. For a casual rider like me, this makes me more confident about taking the bicycle out for rides because I know if I'm too out of shape to pedal any further I can still get home on battery power.

Saturday, June 6, 2009

Public Key Authentication - Update

Turns out it's not easy to uninstall Ubuntu's encrypted home directory feature either, at least not while logged in over an ssh session. All I succeeded in doing when attempting to uninstall was screwing up both computers so bad I can't log in to them any more, and losing all my files. (Luckily there wasn't much on there yet to lose.)

As long as I have to go to trouble of digging these computers out and hooking up a keyboard and monitor so I can reinstall everything, I'm going to put Debian on them instead.

Saturday, May 30, 2009

Public Key Authentication

I haven't been updating this blog lately because I've been trying to get my webserver set up first, so that I could put the pictures there and just link to them from the blog posts. (Uploading pictures to this blog is a tedious process, so I'm planning to create a website that I can just drop a folder full of pictures onto and have them all display nicely. I suppose I could just get a Flicker account instead, but I like doing things the hard way.)

So that I won't have to type my password over and over again to manage my webserver, I took an evening to set up public/private key authentication so I can just use that to prove to my webserver who I am when I log in, and then I don't need a password. Well that was the plan. It ended up taking an evening, then a weekend, then another few evenings, then a week... Every time I thought I had everything set up properly, the next time I came back and tried to log in, the key didn't work. Eventually it got to the point where I'd make a key, it would work exactly one time, and then never work again until I made a new key. (If you know how public/private keys work, this makes NO SENSE. But there it was.)

I spent more time troubleshooting this problem than all the cumulative time I will save from not having to type a password in the future. Nothing wastes your time like a technology designed to save you time, right? But I'm stubborn and the longer this went on the more determined I got to figure it out. See, I have a theory about life: success or failure is self-reinforcing between your beliefs and your actions. If you believe you can't make technology work for you, then when faced with a problem you'll give up too soon. By failing you proved right your own conviction that you would fail. Because you'll never know if the right solution was miles away from what you attempted, or whether it would have been just around the corner. On the other hand if you maintain an unshakeable belief that eventually, something you try will work, eventually it does, and you vindicate the confidence you had in the first place.

Turned out the problem in this case stemmed from an innocent decision I made when installing Ubuntu Linux on the server. In this new version of Ubuntu, they added a feature for encrypting your home directory. The install program asked me if I want to do this, and explained that it would be perfectly transparent to me; the operating system would magically decrypt and mount the home directory for me when I log in, and safely unmount it when I log off. Totally convenient - I'd never notice anything different. "Why not?" I thought. It didn't sound like much trouble and who wouldn't want more security on their webserver? So I left it enabled. BIG MISTAKE.

I would never have guessed there would be an interaction between this feature and using public/private key logins until I was looking at the ssh logs to try to figure out why I couldn't log in with my key. At first I thought I had the wrong log - why were there messages from the home directory encryption service mixed in with my ssh logs? And why was it not finding the key sometimes? Wait wait the order of those operations doesn't look right - shouldn't it be decrypting the home directory before looking in it for the key? Ahhhhh... Dammit. Now I get it.

Here's what was going on: the way public/private key ssh logins work is the server looks in your home directory for your public key to use to recognize you. But your home directory is encrypted. It gets decrypted for you when you log in. But you can't log in until it uses the public key to authenticate you, which it can't pull out of the home directory because it's still encrypted. "There's a hole in my bucket, dear Liza, dear Liza..."

You can verify if you've got the same problem as I did: as long as you're logged into the server in one terminal window, public/private key authentication will work in a 2nd terminal window. Log out of all the sessions, and now you can't authenticate anymore with your key. Log back in, and now your key works again in other terminals too. If this works, send an angry email to Canonical. Tell them I sent you.

The moral of the story is don't use the encrypted home directory feature, it was released half-baked. Have you heard the expression "Beware Greeks bearing gifts?" The meta-moral here is beware operating systems bearing gifts of "features" you didn't need!

Friday, April 10, 2009

Final Days Before Robot Competition

The Hardware is (mostly) done but as usual Software is holding up the show. Lateness is the nature of software; it's worse with robots because given the choice between finishing the hardware and finishing the software, you work on the hardware first: the software can't get around without a body. Then you try to get all the software done in the final 10% of the schedule. I told my team (only half joking) that the first time it runs the whole course, if it runs it, will be at the starting line at the competition!

Testing the GPS is particularly challenging because you have to be outside in a big area to test it well. It's been unseasonably (brutally) cold here some days, mixed with many other forms of inclement weather, so I haven't wanted to walk around outside with a circuit board and laptop, especially at night which is the only time I have to really work on things. Luckily I'm off work today on a three day weekend; I'm going to need every minute of it.

When I did a cursory test of the GPS in a park several weeks ago, I tried to test the minimum resolution of the GPS by standing in a location, and walking a few meters to see how much it changed per every meter. That proved to be futile, so I tried standing in one place, walking several meters away, and then returning to where the GPS coordinates said I should be in the same starting location. Inevitably, I ended up nowhere near where I started, even though the coordinates were the same.



Today I realized that if you leave the GPS unit facing a window long enough, it will eventually get a fix, even though you may be indoors. The picture above plots all the results returned by the GPS over a 20 minute period. (Use the cars in the picture to judge scale.) My incorrect assumption had been that the GPS might be off by several meters, but that you could compensate for that and get a stead progression of readings. The problem is that the amount it is off varies from one reading to the next, even if you are standing still. What you really get is a cluster of points near your real location. Now, the center of that cluster as a whole might be off a fixed amount, and you can compensate for that a bit, but you can't take any individual reading and apply a fixed offset and say "ah, this is where I am". It's more like darts shot at a dart board. Also, the cluster as a whole eventually shifts over time - twenty minutes after this screenshot was taken, the GPS began giving readings further down and to the right.

So, where does this leave our navigation algorithm? Since the data is statistical, maybe our control routine should be as well. Instead of doing 90 degree turns and full stops every time a new reading puts us 10 meters left or right of where we thought we were, maybe lots of readings should have an aggregate effect on the direction of travel. For instance, if we're heading N, the first time we get a reading indicating we should really got E, we won't head directly east but instead shift our heading slightly NE. Maybe the next reading comes in affirming a northerly course, at which point we head N again; it would not be until several readings confirm "go east" that we completely turn N -> NNE -> NE -> E.

Oh and did I mention the compass sensor that tells us whether we actually are going north or east is also not 100% accurate? Guesses stacked on top of guesses...

Tuesday, February 3, 2009

Giant Robot Headed for Colorado



Woot! We're going to Colorado! Sparkfun.com is hosting an Unmanned Autonomous Vehicle contest in their parking lot in April. This is exactly the kind of contest my powerwheels robot is designed for!

When I first read about the competition, my first reaction was "of course we never have anything like this in Oklahoma." Then I thought, damn it, after all the cool robot events I miss out on because they're in Seattle, or California, I'm sick and tired of getting left out just because I live in the sticks. I'm going to this one. Colorado isn't that far.

It's funny because you could see the same thought process go on when I asked my friend Earl Martin to come with me. "Oh I'd love to go, but really I can't..." It was his wife who convinced him, "Go! Go have fun!" (Wives of geeks, take note: you may not be able to break us of the habit of taking over the kitchen table with an occaisional soldering project, or the habit of accumulating mountains of computer electronics junk, no matter how often you roll your eyes at it, but a just little bit of encouragement from you goes a long way to making us feel enthusiastic again when we need that approval. I'm indebted to my own wife for selflessly sacrificing our living room so I can have a place to work on the robot to get it ready for the contest.)

I also asked my friend Ben to come along. He's a great guy to have around, but he's hard to pin down. When I wanted him to be one of my groomsman in my wedding, he was off in England! So I don't know if he'll be available for the trip but I hope so. He said he might be able to fix me up with a mini-ITX board running Intel's open source machine vision library - very cool!

My sister Ginny also asked to come along. I had automatically assumed it wouldn't be something she would be interested in, but when I mentioned it, she asked to come along. I don't know how the transportation situation will play out because my car doesn't have a lot of room, and most of it will be taken up by the robot, but I'm sure things will work out. If she wants to go I certainly don't object and who knows? We might turn her into an engineer, by osmosis.

Thursday, January 22, 2009

Meatloaf Recipe

This is a meatloaf recipe that sort of just evolved out of cooking experiments done by my mother and myself with the help of my brothers and sisters when I was a teenager. We (especially my mother and I) are not big on measuring, so everything is approximate, guessed on what looks good, and seasoned to taste. Sometimes it comes out really good and sometimes really awful, but either way, we never make a bland meatloaf!

In a large pyrex bowl, crush 3 large handfuls of Kellogg's Cornflakes to powder using your hands.
Add:
1/4 teaspoon black pepper
1/4 teaspoon salt
a dash of parsley
a dash of sage
1/2 teaspoon fresh thyme
1/2 teaspoon fresh rosemary

Mix these dry ingredients together.

Then add:
2 cloves minced garlic
1/3 of a whole onion, chopped
1/3 of a whole green pepper, chopped

In a separate bowl, beat 2 whole eggs
Add the eggs and vegetables to the dry ingredients in the pyrex bowl, and mix

After mixing the other ingredients, only then add the hamburger. How much hamburger? I dunno - enough to fill the bowl! The key to making good meatloaf is to handle the ground beef as little as possible. It's already been ground up, and if you mush it around too much mixing it, you'll end up with a meatloaf the consistency of canned cat food. A good meatloaf has the same texture as a grilled hamburger patty.

To mix the meatloaf right, first pull the hamburger apart while breaking as few "strands" as possible. Then fluff the mixture almost as you would if you were mixing butter into popcorn. The goal is distribute the seasonings and vegetables throughout the meat, not to homogenize it. Don't push the meat back together as you mix it; we'll take care of the density by packing it down when we actually put it in the pan.

Place the mixed meatloaf in the pan, and pack it down as solidly but don't smush it. Set aside.

In another bowl, mix the SECRET sauce:
1/2 cup ketchup
1 heaping teaspoon brown sugar
two dashes of chili powder
a hint of nutmeg
a heavy dash of black pepper
1 clove of garlic, ground to a paste
a sprinkle of fresh rosemary, crushed and torn
one drop of yellow mustard

The sauce - I just made that up. Tastes great. Tangy, like teryaki sauce. Melissa's grandmother had suggested the brown sugar and mustard.

Pour the sauce over the top of the meatloaf and spread evenly.

Preheat oven to 350 degrees and bake 1 hour or until firm. If it isn't delicious, don't blame me - you must be doing it wrong. :D