Propeller VGA-lyzer Part 2

The circuit for this is pretty simple - QX-10 video in one end of a Propeller protoboard, VGA out the other.  There's some resistors to protect the Propeller inputs.  Since the QX-10 sends 12V power down the video cable, I can power the Propeller board on the QX-10 video cable alone (the white connector on the right).

As it turns out this circuit is too simple, because I have the pixel input going straight into the Propeller with no preprocessing.  I accidentally dropped a zero in my back of the envelope calculations about the input data rate.  I estimated the pixel clock to be about 1.44 MHz, which can easily be sampled by a 20 MIPS Propeller core.  In fact the pixel clock is going to be in the 14.4 MHz range, and you need more than 20 million instructions per second to read and store that many pixels per second (you have to burn some of your instructions budget to shift, store, and loop).

There's various tricks I could do with the Propeller to get around this.  Some existing Propeller video drivers use as many as four cores working together to achieve higher video output resolutions than normally would be possible.  I could use a similar technique here to share the sampling load among multiple cores.  However going to such software gymnastics to save on hardware seems like it would make it harder to debug than is worthwhile.  Since there's no penalty for reading in data 8 bits at a time rather than 1 bit at a time, I can get a free performance improvement simply by sampling the pixel data into a 74HC595 8 bit shift register on the front end before consuming it in 8 bit chunks.  This will provide a second tangible advantage over the multiple-cores method:  with a shift register on the front end I can skew the phase of the pixel clock independently of, and more finely than, the microcontroller software instruction rate.  That's important because if you sample pixels at the right rate but the wrong time you'll get shimmering.  Optimizing the phase of the pixel sampling clock with respect to the video signal will get the best picture.

However that will be a future revision of the circuit; rather than make that hardware change immediately, I decided to see what I could accomplish with the hardware I have built.  (Who knows maybe I'll discover an additional tweak or feature I need to make to the hardware, so I might as well find out what I can.)

Although I will certainly need to use assembly language for the final product, I chose to use the interpreted language Spin for preliminary exploration, because it will be easier to debug.  Even though Spin is woefully slow compared to the video data rate, it does still allow accurate and precise wait timing, and there is a trick you can do with precision timing to capture a video frame slowly over time.  The idea is rather than try to capture the entire frame at once, you wait a precise amount of time after the sync signal to capture a certain single pixel from a line, then you have to wait an entire frame to capture the next and so on.  It is slow but it will work for static images.

Speaking of debugging, I also found that an important debugging technique is to have the software toggle one or more pins at certain times in the loops.  For instance originally I tried the slow-capture trick based off the horizontal sync signal, but when that didn't work I suspected my Spin code might be too slow for the 19kHz signal and may be dropping syncs.  By having the code toggle a pin every time it captured an hsync I was able to see with the scope (actually with its frequency counter feature) that indeed the code was indicating fewer hsyncs captured than expected, and it was unstable.

After discovering my Spin code could not keep up with the horizontal sync, I swapped the nesting of my "x" and "y" loops and switched to basing all of my video capture timing off the much slower vertical syncs.  Doing this with only vertical sync is 400 times slower than even the slow horizontal sync way.  I am able to read just one pixel per frame.

I was at last able to capture something from the QX-10 video signal!  Since it takes so long to capture a frame, I tweaked the code to concentrate on just the section of the frame where something interesting is onscreen, oversampled it and blew it up.  The image below is 2x super-sampled and 2x to 3x magnified.  I can't quite be sure, but I believe through the terrible mess of jitter and noise in that frame capture I can make out the message "INSERT DISKETTE".  (The E's are suggestive.)

I'm a little disappointed that the image isn't clearer, but I guess I should be amazed that it even works.  Because what is happening in this picture is that it is taking a single sync signal for the whole frame, waiting a calculated period of time that has to be accurate to an average of 1 part in 160,000, and capturing a single pixel out of hundreds of thousands.

Propeller VGA-lyzer Part 1

My first computer was an Epson QX-10, which is an old monochrome green screen 8-bit machine.  The unit I had is long gone, but I picked up a replacement on E-bay some years ago.  I got a good deal partly because it lacked a monitor.  (The QX-10 monitor used a proprietary connector and proprietary signaling scheme, so you can't simply plug in a modern - or even an old - PC monitor.)  I was not too concerned about this, since I had tinkered before with video signals in Earl Martin's Industrial Electronics class at Vo-Tech, including a similar proprietary-video-to-PC-monitor adapter project that was successful, albeit with a different video signal and analog CRTs (now I would use an LCD monitor).

Having occasion to reorganize my home office / lab and to pull this among sundry other computing curios from storage, now is opportune to tackle this video conversion challenge.  Like most of my projects I started off assuming it would be easy and it has turned out not to be so, but I am also learning more than I expected to.

An obvious first step is to make a simple circuit that reconciles the merely electrical differences between the mystery signal and VGA spec, to say nothing of timing, and just hook up a VGA multisync monitor to see if it can lock onto the oddball timing.  In fact this was the approach that worked for the Vo-Tech project many years ago, so that I never had to go on to the next step of dealing with sync timings.  (I think I just had to make a passive R2R D/A circuit to integrate the colors or something, and I passed the sync signals straight through.)  Well I say "worked," actually the first CRT monitor I plugged that Vo-Tech project into self-destructed with the bang and acrid smoke of blown capacitors.  A lesser engineer might have doubted his circuit prototype, but I theorized the problem must be that the cheap, used monitor I'd tried it on couldn't handle the high resolution input and had burned itself up trying.  If I just connected my circuit to one of these brand new, large, expensive monitors that belonged to the school, surely it would work there.  Fortunately for my instructor's classroom budget, I was proven right and it did work then.

For the current project I looked at the sync signals in the scope to figure out what I'd need to do them electrically.  VGA is a positive-sync video standard and I could see in the scope that waveform pattern inverted in the QX-10 syncs, so I ran the sync signals through a TTL inverter chip before connecting them to the monitor.  (For the record you can also easily tell which sync signal is which by looking at the frequencies, but I had a wiring diagram to tell me that too.)  I don't think I'd connected the color signal(s) yet, but if I had I could have used a voltage divider between a resistor and a forward biased diode, since the QX-10 monochrome signal is 0-5V and the VGA colors are 0-0.7V, and I don't care about intermediate intensities just on and off.  However, after I hooked up the sync signals I could see it wasn't going to work; the LCD monitor I used was already displaying the message "Cannot sync to this video mode."  Perhaps LCD monitors have lost the multisync capability good CRTs had, or perhaps it's because I'd connected my prototype up to an old, second hand LCD monitor.  I decided not to bother to try connecting it to my own brand new, large, expensive LCD monitor.

Right, so, onto sync timings, frame buffering, and phase locking!  I will be designing "real" video converter hardware for this after all.  Here's the situation.  Completely opposite to that Vo-Tech project, the big problem with the resolution / sync frequencies from the QX-10 is that they are too slow.  Well actually the QX-10's screen resolution in pixels - 640x400 - is coincidentally the same as a VGA standard resolution, but it does its vertical refresh at the leisurely pace of 45 Hz.  (Normal would be about 60 Hz.)  This in turn makes the horizontal sync frequency lower, and the pixels clock out slower.  Thus even though the number of horizontal syncs per vertical sync is "right", the frequency at which they happen is too slow.

This difference in vertical sync refresh rates makes buffering the entire frame the obvious and really the only option.  See, if it were merely a problem of not enough lines (not enough HSyncs/second), I could buffer one line at a time, and sometimes play back the same line twice in a row to spoof a higher number of horizontal lines than the input really has.  But there's no way to fake it with a vsync mismatch; one can easily see mathematically (45-60=-15 and 45/|15|=3) that every three seconds you'll build up such a deficit of lines that your output will demand a line an entire frame away (and the new one hasn't been transmitted yet).  I am not really an expert on video conversion, but I think I can say if you have a v sync mismatch you must to buffer a whole frame.

That's ok.  This is what I wanted to do anyway.  I need 32K to buffer a whole QX-10 video frame, and my favorite microcontroller, the Propeller, has exactly 32K RAM.  I wanted to be sure I wasn't missing an obvious way to do it with less RAM, so that the premise of the project isn't invalid.  Once I have this working for the QX-10, the Propeller object I write for this might be useful for connecting all manner of vintage computer and video game systems to more modern monitors or televisions.  That's why I'm calling this project the VGA-lyzer.

Freerunner Battery Mod, Case Mod, Runs Android

Oklahoma was hit by the biggest snow storm in state history this Christmas. Since we were snowed in and couldn't visit relatives, I found myself with some unexpected time on my hands. So I built my own Android phone.

I bought a Neo Freerunner last year, but some design flaws kept it from being usable to me as an everyday phone: pitiful battery life, audio problems, and no single Linux distribution stood out as the one best choice to use. But the battery was the worst problem.

My previous experiments had shown that if you remove the battery monitor circuit board from the original battery:

...you can transfer it to a larger Li-Ion battery, like this one from a portable DVD player, and it will work. Surprisingly the BQ27000 chip on the smart battery board is able to learn the larger battery capacity.

Of course, I didn't let the old battery go to waste either...

The prototype battery mod worked but I needed to give the phone a bigger enclosure so the battery wouldn't have to be on the outside where it might be damaged.

Having proved the concept, I exchanged the junk DVD player battery for a brand new 6 AH Li-Ion battery from Sparkfun, and put it all into the case from a 2.5 inch portable hard drive. (To put that in perspective, the original battery was only 1100 mAH.)

Rather than try to cut the hard drive case to perfectly match the Freerunner circuit board, I simply reused the original phone faceplate since it already had the proper board mounting locations. I cut an oval hole in the hard drive case (free hand with a dremel), and then used JB Weld to join the two pieces and to even out the gaps between them. I put the JB Weld on sloppy but used a wet paper towel to wash away the excess, leaving a nice even seam all the way around. (Turns out JB Weld is water soluble - who knew!)

By the way that silver squiggle shape on the front is the cellular antenna. I ran out of room on the inside, so I put the antenna on the outside! And putting the antenna next to my cheek is a good way to extend a sarcastic "bite me" to all those idiots who argue from a position of ignorance that cell phone signals cause cancer without knowing the first thing about the physics of radio signals.

Although from the faceplate it may appear as though an entire Freerunner has been simply embedded into the new case, behind the faceplate there is nothing left of the original Freerunner case, and some components have been moved and some improvements made as I tried to rectify my main complaints about the original phone.

Here it is charging at a total rate of nearly 2 amps using both the fast charge board and the Freerunner's internal charging circuitry. No apparent harm comes of running them both at once, and the extra charger doesn't confuse the battery gas gauge chip either.

The BQ27000 learns the battery capacity by observing a complete battery discharge cycle. Unfortunately software will shut down the phone when it thinks the battery is empty, so we'll never actually reach a full discharge of the new battery to recalibrate the gas gauge! Luckily I built a battery discharge device a while back, which is really just a mess of enormous low-ohm power resistors strapped to a massive heatsink with a fan on top. And a fancy digital temperature readout so I know whether I'm actually frying my power resistors. This discharges this battery at a rate of 1 amp, which is a bit too fast, but with a 6 AH battery I could be waiting all weekend for it to discharge at "normal" current draws.

The documentation for the BQ27000 gas gauge chip mentions that it doesn't revise its capacity estimate downward more than 1/8th of the total at a time. It doesn't say anything about how the estimate gets revised upward, but experimentation with the test battery from the DVD player indicates that it caps the upward change at 1/4th of the previous estimate each time. The test battery reached 2.2 AH, so I can only expect the BQ27000 to revise the estimate up by about 0.55 AH on this cycle. It's probably not worth the trouble to work it up to the full 6 AH reading, since the process takes 12 hours and I'll be slightly damaging the battery every time I completely discharge it! Maybe when the battery is deteriorated in a few years, I'll calibrate it to whatever reduced capacity is still in it.

Not really visible in the pictures, but I removed the speakerphone (which was useless in its old location because its proximity to the microphone made it impossible to actually use speakerphone without getting a loud feedback squeal) and connected it in place of the earpiece. I thought that, because it was bigger, it would be louder than the earpiece, but instead it is much softer, nearly inaudible. I think that's because the earpiece measures as high impedance while the speakerphone speaker is low impedance, so there isn't an efficient transfer of power. Tomorrow I'll cannibalize the audio amplifier IC from that same portable DVD player I got the test battery from, and use it to fabricate a tiny audio amplifier board to drive the earpiece & match the impedances. (I can design a circuit in my head and build it on a new board as fast as I can take the components off the board I'm cannibalizing.)

That will also provide me a physical volume control so I won't have to rely on the goofy software volume settings that never seem to work right on the Freerunner. Hmmm... Maybe I'll make it stereo while I'm at it. The Neo 1971 (prototype to the Freerunner) had stereo speakers in it, but not the Freerunner. The audio amplifier chip in the DVD player is stereo, and I can pull a stereo signal from the headphone jack. Interesting...

So... Android right? Remember at the top I said that my original goal was to address some specific flaws with the Freerunner, one of which being lack of a definite distribution to stick with. I didn't actually like Android when I first tried it (still not sure about it) but it seems to be where the ball is rolling. When I began to think I wanted a Motorola Droid, and when blogs and podcasts started going crazy over the (latest) rumored Google phone, I had to stop and remember, wait a minute, I already HAVE hardware capable of running Android. It's just been sitting on my desk in pieces waiting for me to put it together.

Installing Android was a breeze; easier than any other install I've done on the Freerunner. You just untar a file onto an SD card, and then booting off the SD card prepares Android on the Freerunner. No messing around with the flashing the phone over USB, and no messing around with bootstrapping the system on the phone itself like Debian does. Just pop in the SD card and it just works. Not much else to it - except wifi doesn't work. Why does wifi never work for me on Linux? Oh well...

Anything That's Not Tied Down

This morning when I got out of class I found someone had undone several of the velcro straps that hold my electric bicycle's battery on. Fortunately the battery (and the bicycle!) were still ok.

My guess is someone just saw a bulky black bag and decided to steal it without knowing what was inside. Walk-by snatch and grab. I like to think the sheer weight of 30 lbs of lead acid batteries scared them into letting go of it. And anyway there's more straps holding that thing down on the front side.

Everything Takes Longer In C++

So I had tonsil surgery last Monday. Even though I didn't feel up to programming for my day job, I couldn't stay away from trying to work on the MMO game project called Emergence that Ben and I are creating. When it seems like progress is going glacially slow on our game Ben and I always say "if only we had more free time to work on this..." Well I've had time this week - and I still made only baby steps in progress on it. Granted I was resting from surgery and on pain medicine, but still... I'm beginning to think it's not (just) that we have no free time. It's that with programming you put in enormous effort yet make only incremental progress.

If it takes a certain amount of effort to write software of a given complexity, writing software that's twice as complex doesn't take twice as long, it takes, I don't know, ten times as long, twenty times as long. The effort increases exponentially. Invert that and you get an equation that says that you only get logarithmic increases in output for every increase in effort. If you don't know what a logarithmic curve looks like, it has a knee bend. If you stay to the left side of the curve you get pretty good output for input, but as you move right it levels off quickly. To the right side of the curve, increasingly large increases in input result in increasingly small changes in output.

So I think I've been working dangerously close to right side of the curve lately, where what you can accomplish tapers off faster than the increase in effort. I've been getting better with some of the new programming techniques I'm using, and I've streamlined my build process and improved my debugging techniques. But here's the funny thing - all those improvements really just help me increase my work input. They don't directly increase the output. You see if I streamline my build process and my debugging techniques so that I can make a change and test it in half the time, that just means I can test twice as many code updates in the same amount of time. But if I'm on the inefficient side of a logarithmic curve, I may actually need to make four, or ten, or twenty times as many code changes at a time to actually double my output.

So that's what's so frustrating about this work. You run faster and faster but never seem to get anywhere. It's not hopeless though. I think the fact that the ratio of accomplishments from effort is logarithmic is some kind of fundamental law of software, but it is possible to tweak the shape of the curve. Although it will always eventually taper off, it doesn't need to taper so as quickly. I think changing the programming language you program in does the most to change this curve. C++ is very powerful and generates efficient programs, but it has a very sharp bend in its complexity curve. This is because doing anything substantial in C++ just involves so many fiddly little details, and the language does so little for you automatically. I'm not saying you can't build large systems in C++, but I am saying it probably took exponentially more work to do it than it would have to do it in a scripting language. And I'm not saying you shouldn't use C++; in order to get good performance in our game, we need to. But we're going to use both C++ and script language in our game, and I need to get out of the C++ part and into the scripting as soon as possible.

The ironic thing is that the component which is giving me all this trouble, where I'm working too far on the right side of the complexity curve, is the C++ component that will bind our C++ side of our game to our scripting side. In order to leave C++ behind I need to first overcome a trial by fire of the toughest C++ I've ever worked on.

Electric Bicycle Conversion

A student parking pass at the University of Oklahoma costs ~$200 a semester. As if that weren't bad enough, they first try to trick you into buying the nine hundred dollar reserved pass. Instead of paying "the man" $200 just so I can park in overcrowded, confusingly marked lots, I decided my money would be better spent upgrading my bicycle so I can park a few miles away and bike to class.

I knew about these electric bicycle conversion kits because a couple of CS Ph.D. students I'm friends with, Lawrence Kinchloe and Mark Woehrer, put electric conversion kits on their bicycles a couple years ago. (Hey guys! We should start a club!)

It turns out WeRelectrified.com sells a low-end, but complete bicycle conversion kit for $200. (Ha! Exactly the same as I would have spent on a parking pass!) While I could have fabricated my own motor controller, and scrounged my own batteries, the complete kit is such a good deal that it didn't make sense not to just use it as-is. There's even more than what's in the picture - it also comes with a metal rack to go on the back of the bike for the batteries.

The way this thing works is there's actually a motor inside the front wheel, a big fat metal hub that makes the front wheel turn by itself. (Nothing gets changed on the back wheel - so you can still pedal it too, like a normal bike.) In theory, you can just replace the front wheel of your bicycle, put the batteries on the back and the throttle on your handlebars, and you're ready to go!

That's the idea, anyway. Of course, this is a Project, and as we all know Projects rarely go according to plan. The first problem I had was that the hub is too wide to fit between my front forks. Earl Martin would probably just have grabbed the front forks and pulled them out, but I'm not that strong. Instead, I took the scissor jack that came with my car, slipped it over the front forks, and just cranked it on out there. (Nick Johnson asks "Is that SAFE?" Come on, Nick, it's a steel frame! You could probably safely bend it into a pretzel if you wanted to.)

OK, now the hub fits - but the axle is too big to go in the notches on the end of the forks.

Not a problem for a man who owns more kinds of steel files than socks! This is a delicate operation - the axle has flat sides on it, and I want the notch just wide enough for the flat sides to fit, but not wide enough for the axle to rotate in place. When the hub motor turns the wheel, it's pushing against the axle to do so, and you don't want the axle to be able to start spinning within the forks.

Great! Front wheel fits now!

OK, now to transfer the tire from the old wheel to the new wheel. Except the tire doesn't fit. In fact, these aren't the same size at all! Did I buy the wrong size wheel? Oh, never mind. This is the same thing I went through last time I worked on this bicycle! The wheels I put on it then were narrow 26 inch, which means they have less tire material and more rim circumference than the moutain bike wheels it came with; the new wheel brings us back full circle, with smaller diameter rims under thicker tires. Means I need to buy a tire.

Off to the bicycle store!

Since the front wheel is now a different size, I decided to change the back wheel to that size too, and buy new tires for front and back. I've been meaning to replace the back wheel anyhow (between either the wheel starting out warped or my asymmetric weaving pattern, the wheel was getting ever so slightly out of round). I didn't think I needed to buy new gears though because I could just take the gears from the old wheel, right?

Oops... that's not how you remove the gear pack. (If ball bearings fall out all over the place, you're doin' it wrong.) If I read my own damn blog from last year I MIGHT have remembered that. I never was actually able to get the center portion of the hub off.

BACK to the bicycle store!

Another one hour drive. This time I came back with a gear pack. I put yellow teflon tape on the threads of the new back wheel - hopefully if I ever need to remove the new gear pack, this will keep the gear hub from getting stuck as badly as the old one did.

Then I remembered that on the second trip to the bicycle store, I forgot to return the inner tubes I bought on the first trip. The guy at the store mixed up the two sizes of 26 inch just like I did with the wheels, and sold me the skinnier, larger circumference narrow tubes when I needed the wide 26 inch tubes. These tubes don't fit in the tires I bought (there's excess tube), but by this point I'm beyond caring so I just jammed them in there anyway, kinks and all. I haven't gotten a flat yet so I guess it's OK.

I'm kind of proud of the way the throttle mount turned out. I didn't want to unwrap all my handlebar tape to slide the throttle assembly on, so I wasn't sure how I was going to mount it. Eventually I realized it would just as easily fit on a piece of 3/4 inch PVC pipe, so I mounted the throttle vertically on a piece of pipe. I knew if a convex pipe edge butted against a convex handlebar edge it would never stay in place, so I cut a half-circle indention in the side of the PVC pipe where it mates to the handlebar. The mounting feels solid and the throttle is not any harder to use in this position.

It has a key for an on/off switch. It happened to be a perfect fit for this space under the bicycle seat. While it might look a little rude if I reach between my legs to turn the key, I figure this will keep the rain off it.

I mounted the motor controller box upside down under the battery, and I made all the wire connectors come out in the space hidden between seat and the battery bag. Hopefully that will make the wiring less obvious from above. Although I'm not doing anything wrong, I don't exactly want to advertise that something strange is going on. My experience with authority has been that even harmless things will be treated with suspicion if they out of the ordinary. Besides, I don't want to give potential bicycle thieves any reason to think my bicycle is special.

Here's the completed bicycle. As long as I had to buy new tires, I thought I'd have a little fun and get ones with a red stripe. I think it's a nice effect. You can see the batteries here in the unzipped bag; it came with a block of styrofoam in the back where I assume the 3rd battery would go if this were a 36 volt kit instead of 24 volts.

On my first day riding it to class, the zipper failed on the bag where the corner of the battery meets it. The batteries, which are sealed lead acid (heavy as hell but cheaper than lithium ion), rest directly on the metal frame of the rack. There's no padding in the bag, so with every bump the batteries slam down against the rack and then up against the zipper.

Catastrophic zipper failure!

When this happened, I wasn't even sure how I would secure the batteries for the ride home. I actually have a pair of shoes that I've threaded two sets of laces into for MacGuyver emergencies. I could have taken out one set of laces to tie down the batteries and still have had the other set of laces to keep my shoes on. Wouldn't you know it but this would be the day I wore a different pair of shoes though? Darn the luck. Fortunately I had previously fixed the water bottle holder with baling wire, and there turned out to be enough when I unwrapped that to hold the batteries down.

The foam from this padded laptop bag will be just what I need to cushion the batteries. After I cut out the foam, I'll fold it in half and put it under the battery bag.

A zipper is completely off one side. How will I get back it on track? By cutting deeper into the throat of the zipper, I was able to give myself enough slack to re-thread it from the end.

Then I made the two zippers meet at the corner where the original tear happened, so I don't ever have to cross the damaged section! The amount of offset I created from rethreading the zipper on the left side happens to the same length as the stretched portion here, so everything matches up properly.

The next time I have something go wrong with the bicycle, I'll have some duct tape stowed onboard. I removed the hard cardboard center from a roll of duct tape so I could fold it up. I think I saw that trick on Instructables.

And there it is, back together and nicely padded.

Unfortunately the cheap made-in-china 24V battery charger that came with the kit died after just one use! I can use my car battery charger to recharge the batteries, but to do so I have to change the wiring on the battery pack from a 24V serial configuration to a 12V parallel configurations every time I hook up the 12V charger.

Finally done, fully charged and put back together. Despite the problems, I would say the project has gone better than I anticipated. It's lots of fun to ride and completely practical. I think if more people understood that we'd see as many electric bicycles in stores and on the road as regular ones. It still works as a bicycle, so you can rely on electrical power as much or as little as you want. For a casual rider like me, this makes me more confident about taking the bicycle out for rides because I know if I'm too out of shape to pedal any further I can still get home on battery power.

Public Key Authentication - Update

Turns out it's not easy to uninstall Ubuntu's encrypted home directory feature either, at least not while logged in over an ssh session. All I succeeded in doing when attempting to uninstall was screwing up both computers so bad I can't log in to them any more, and losing all my files. (Luckily there wasn't much on there yet to lose.)

As long as I have to go to trouble of digging these computers out and hooking up a keyboard and monitor so I can reinstall everything, I'm going to put Debian on them instead.