Well, I've run into 2 big problems.
First, is that although I thought I had all the scanline jitter solved, it turns out that I had a few places where there was still jitter. And this has been really hard to track down. After a few angry evenings staying up too late debugging, I finally realized that it wasn't actually my display kernel causing the problem (where I had spent the longest time debugging), but that occasionally I was taking too long in my main update code, and not rendering the screen in time. So I need to do some combination of optimizing, removing code, or running updates every other frame.
The other problem is harder to deal with. I asked a friend if he could test with his Harmony Cart (an sd-card-based atari cartridge to run on a real Atari), and the game doesn't work. AT ALL. It won't do anything, not even display anything on the screen. So something's really wrong. Unfortunately, I have no idea what. At this point, I think the only thing I can do is get my Atari up and running, buy a Harmony Cart myself, and go through my git history testing different builds. At one point, about a year ago, I know that it ran. I need to find out where it stopped.
So I got the Atari out of the garage last night, and went through about 8 or 9 combinations of adapters and cords trying to get it to work. (The RF switch box that came with it is somewhat broken. I tried with my NES RF box, SuperNES switch box, and a few other gizmos from my box of adapters. I had the most luck with the NES switch) The best I got was an extremely blurry screen that you could somewhat see. Not enough to really enjoy playing on. Now do I want to spend $60 on a special cart just to debug this on my terrible TV?
I have a few more things I can try to get it working better. If I'm going to spend the money on a flash cart, I want it to at least look good. We'll see what happens.