Well, I bought a new RF adapter, soldered on a new controller jack, and bought a Harmony Cartridge.
With that in place, it only took about an hour to solve the problem:
Going through the process of starting with ridiculously simple sample code, testing on the Atari, and then slowly adding complexity until it broke, I finally narrowed it down to a relatively small subset of places that the problem could be.
It turns out that I was trying to do some fancy bank jumping acrobatics before properly zero'ing out everything. Ok, boring explanation time:
When you turn on the Atari, the entire block of ram, and the state of the CPU registers, are completely unknown. So the first thing you should do is start zero'ing them out.
Now one of the things that's unknown is also what memory bank you're running in. Which means that although you know what memory address the program starts at, it could be in any bank. So one of the first things you need to do is, in the same place in every bank, run code to jump to the bank that has your actual program beginning in it.
Jumping banks is as simple as writing to a magical memory location, BUT that just immediately switches banks, leaving your program counter (ie memory location of the next instruction) as it was. So you have to make sure everything lines up nicely. I previously explained how I made some general routines for doing bank jumps, which involved writing an address to the stack, then issuing an RTS command, (return from subroutine), which makes the CPU pop an address off the stack and jump to it. And I was using those for my initial beginning-of-program jump to my actual code.
Well, it turns out that if your stack and stack pointer are all in a broken random state, you can't reliably write an address to the stack and return to it.
The awesome Stella emulator has some settings to randomize memory on startup, which helps you generally find these issues, but I guess it wasn't randomizing something quite enough. Because once I properly zeroed out and initialized everything BEFORE doing my bank jump, everything worked!
Now I need to figure out why the colors are so completely different....