More importantly, I thought all of the smart people were developing software this way.
Chapter 1: Starting at NCSA
My first job in software was in college, working support for NCSA Mosaic. To this day it’s still one of the biggest breaks I’ve ever caught, and it set a tempo of my career for at least the next ten years.
My first two summers I went back home and worked at the best bicycle shop in town, doing assembly and repairs. I learned a lot about craftsmanship, good tools, and how to take care of your workspace, though it took a few more years for it to sink in fully. But by junior year I felt like I needed to be more independent and took a year sublet on campus with some friends of friends. I was a young man in a terrible hurry, and I felt that Junior Year was the time to start a job as a student programmer, so I went looking.
Things were looking pretty bleak. My first interview was terrible. I loved concurrency and networking and there was a research project trying to build concurrent extensions to C++. I had almost no experience and was hoping that enthusiasm would suffice, but when the day came I blew it. I slouched. I wore shorts (It WAS summer in Illinois, and they were my nicest shorts, but they were still shorts), and I’m pretty sure they were hoping for a grad student.
So it was about 5 weeks into the summer, and I had no full time job. Due to seniority and my inability to be anywhere by 8 am, I only had one shift a week working in the computer centers, plus whatever I could pick up covering absences. I was living off of savings – savings that was supposed to be tuition money – and subsisting predominantly on vermicelli noodles, bulk olive oil, rosemary and parmesan from Sam’s Club. My father was understandably concerned. He started saying things that sounded a bit like ultimatums.
I started showing up at the Student Employment office, and one day someone had posted a laser printed sheet of paper, on the job board, eye level, right in front of the elevators. The opening was for a student, doing support for NCSA Mosaic for Windows. I only vaguely knew what it was (not for lack of trying by my friends). It wasn’t a programming job, but I knew it was a big deal and thought I could hack it. I also thought if I could get my foot in the door I could become a programmer. I knew Windows tech support, and email, and I could write, so I came in and did my best tap dancing and got the job. If memory serves Terry had me start either that day or the following morning.
Now if you’re keeping score at home, this is only a few months after Marc Andreessen famously convinced (some would say poached) a group of NCSA employees to quit their jobs and move across the country to work with him on a commercial web browser. There were a lot of new faces working on the project, and the user base was still growing. Rapidly. My friend Ryan Grant had transferred in from another project and had been a developer for a couple months. For fun I didn’t tell him I got the job, I just showed up for work and loved the double-take when he finally noticed me sitting around the corner.
I learned a bunch really fast. For one, I had to be able to hand-parse HTML like nobody’s business, because most questions were about why pages weren’t rendering (sometimes it was our bug, mostly it was bad HTML). For another, I learned I had one of the highest paid student jobs on campus. Thanks in part to Marc, his compatriots, and an effort by NCSA to keep other people from following their lead, I was making $.50 an hour more than you on day one, unless your job was legitimately dangerous. And it only went up from there. I had my tuition money and a bit of spare time to boot, because I didn’t have to work 40+ hours a week if I didn’t want to. Among other things, I coded as a hobby.
My boss had hired me because he needed to spend more time doing managerial things and less time answering email. They were behind on email, and it only got worse as time went on. Almost immediately I discovered that I do not enjoy solving the same problem twice. When I noticed that a lot of people were asking roughly the same questions, I put extra effort into a response and then saved a copy off to a directory. Because we were all using the same email account, this turned into a library of canned responses pretty quickly.
Every week I would suggest something new. Pre-screening. Categorizing messages. A vacation message with links to self-help pages. Looking for replies from people saying they’d solved their own problems. Bankruptcy (Starting with the latest, and throwing out all messages older than a certain time). Participating on Usenet so that we could troubleshoot problems faster, and so others could read the archive. Terry hired a couple more students and left me in charge of them. We would get close but never actually catch up.
It turned into a Red Queen problem. Every time we brainstormed a new way to answer email questions faster, it would get cancelled out by the larger and larger user base. Eventually this lead to the first piece of production software I wrote. Mosaic had its own email client, but if you didn’t follow directions it was easy to send us email with an invalid or nonexistent return address. I wrote a csh script that would look at our bounce folder for permanent failure messages and create a black list of email addresses that we would filter into a separate mailbox and scan through it once a day (I was always worried that a customer we couldn’t reply to would identify a really critical bug). From all modes of failure, almost 10% of our replies would bounce, and so I cut our workload by 10% with half a week’s work.
At some point, fairly early on in this process, it occurred to me that the only way we could stem the flow of email is if the ‘support’ team got into the business of preventing emails in the first place. Many of our users had the same problems, and a lot of them could have been detected before we ever shipped software. I convinced Terry that we really were QA in addition to Support, and that we should put more energy into the Assurance part of that job. So we started signing off on builds, then I started showing up to design and planning meetings.
I found a deep respect for the testing process that most developers have to learn slowly (and some not at all). Thinking of all the ways a piece of code could misfire is really hard, but essential to making a quality piece of code. And it’s better to find a problem early than in production, or after you’ve switched to something else.
Then Netscape version 1.0 was released, and everything went to 11. For me, this was the First Browser War. Marc A. and Eric B. might argue (and if memory serves, Eric does) that Mosaic unseating Cello was the first Browser War, but that happened before people started calling them Browser Wars. The Spyglass founder tells people they were involved, but they were just a blip on our radar until much later. Most people when asked will think it as IE vs Netscape, but if anything that was the Third War.
Both teams were releasing new versions every 4-8 weeks. They would release a feature, and if we liked it we’d copy it. If we had a feature they liked, they would copy it. Because the pace was so breakneck, it got to the point where people on Usenet couldn’t keep straight who was borrowing from whom. We got credit for features they introduced, and they got a lot of credit for things we did. We were mining our email list, Usenet, and the competitor’s applications for feature requests.
I joke that I introduced the stop button to browsers, but it’s sort of true. The animated Mosaic Logo would stop a page load, but nobody understood you could click on it. We got so many questions asking how to stop a page load that I realized we’d failed on UX. So I suggested we add a second button with a big stop sign on it to make it easier. Netscape liked the idea so much they also put in the stop button, and then changed their logo to go to the Netscape home page instead. We stubbornly kept the old behavior (In fact we doubled down and Carl added a much fancier animation of the logo).
We were a team about a third of the size of Netscape, working for a University, and almost – almost – keeping up with Netscape. Because we were free, supported by licensing fees and grants from the National Science Foundation, they had decided to be free as well. For better or worse, I believe that decision as much as anything set a tone for the Dot Com Bubble.
Things were moving fast and we couldn’t keep track of everything. There was a white board in the hall on the outside of Terry’s office. It was right at the ‘L’ between the room the devs sat in and the rest of us. Effectively, if you left your desk you had to walk past this board. You saw it all day every day. We started writing up all the tasks and bugs. If it changed you’d eventually notice even if someone forgot to tell you. Developers would put their initials next to the one they were working on. When they were done they’d draw a box next to it. When we had tested it we’d put a checkmark or X in the box and it was good to ship.
Essentially, this became our Kanban Board, of a sort. Of course nobody I knew had any clue what Kanban was then. We had never heard of Toyota Quality System, and David Anderson, who I would actually interview years later to be my new boss, was still off in Asia Pacific working with Jeff de Luca and trying to dream up TQS for programmers.
We were doing an Agile Development project in 1995 because I thought it sounded like a good idea and was scarily good at asking questions that lead people to interesting places. More importantly, I thought all of the smart people were developing software this way. It never occurred to me that we were onto anything special. I suffered for it, greatly, and at length. Two jobs afterward, I ran into another person who totally understood what I valued. For the next ten years I went around looking for people who didn’t think I was crazy, and in that time I only found two other people I was prepared to call ‘mentor’.
Eventually all good things must end. I had some health problems in the following fall, and did a slow fade for a couple months. I never got to thank Terry for not firing me. I was supposed to be working half time during school, but some weeks I was only doing 10-15 hours or working at very odd times, like 9pm to 2am, and I was too proud to talk to him about it. He knew something was wrong, and he gave me time to sort it out. But external factors soon eclipsed personal issues.
But despite that, the Mosaic experience put a fire in my belly about The Next Big Thing, just as the whole World Wide Web experience put the phrase The Next Big Thing in people’s heads.
Just a few months into 1994, my then-girlfriend sat next to me while I finished homework in the computer lab, and had for weeks been working on her ‘home page’, whatever the hell that was. My friend Ryan, upon transferring into the Mosaic group, had tried to show it to me shortly after they added image support. He seemed pretty disappointed by my “is that it?” response. After I started working there I was a little embarrassed by the whole thing, and I hate being embarrassed.
At the end of my time on the project I had been teaching myself Windows MFC and multithreading by writing an Othello game that kept calculating its next move while you took your turn. The API was so frustrating and I was beginning to despair of what my future was going to look like. So much that I dreamed out loud a few times about finding some people and writing a cross platform UI that didn’t suck. Someone overheard me and asked if I’d heard of this brand new language called Java. Oh and by the way, there was a student programmer opening upstairs if I was interested in getting paid to learn it.
I applied. I didn’t think my interview went very well but I got the job anyway.