My employer has organised a ‘Summer of Q’, where a number of us have signed up to play with AmazonQ. This weekend was the first time I could work with Q in depth. The main result – I ‘built’ a quiz application in 30 minutes (while also doing some chores) and it looked and worked better than what I’d have produced solo. But there are a lot of subtleties and caveats to add to this.
- A major argument against GenAI putting developers out of work is how poor the tooling and signup flows for Q are. The signup is terrible and confuses a lot of people. Q failed to help, and kept hallucinating links to help pages that didn’t exist. The IntelliJ plugin is awful and locks the IDE, so I’ve had to use the command-line version instead.
- Q is great at producing code. Producing the quiz example was a trivial task, so I’m now working on a much more complicated example. Straight away, I can see Q making me more effective. Personal tools I’ve wanted to make, that I decided against investing time in, now look easy.
- The quiz app that Q produced looked and played better than what I could have produced by myself. I’m very impressed by this.
- The model’s reasoning is clever and spooky – it makes mistakes sometimes, but then works to fix those. Interesting behaviour – although I expect there to be fewer mistakes in the generated code over time.
- One of the challenges of coding agents is getting used to the new workflow. There’s a fair bit of waiting involved while Q thinks about each file that needs creating. It’s very different to using a GenAI coding assistant, and I need to figure out the best new workflow.
- An ongoing problem with GenAI is that it involves a lot more reading than writing. I figure almost no-one is reading co-pilot meeting summaries, and I worry that not everyone will closely read the impressive amount of code that Q generates.
- At present, I’m reviewing each action Q takes, rather than trusting it for the session. It’s going to be interesting to how other people are working. There’s a lot of boring waiting this way, but a lot less reading to do in one go.
- Being able to produce decent (albeit not perfect) code so quickly will change the nature of programming. The coding part is going to get much easier. The development part – making sure the right thing is produced – will become more important, and maybe more difficult. I’m currently using feature tests as a way of validating what is being made.
- Something I’ve noticed with GenAI in a number of areas is the importance of taste. The tools produce things (image/text/code) incredibly fast, and require an operator with strong opinions about this output.
- Q responded to my initial, naive prompts by producing ornate additional features. For example I asked it to generate some BDD feature files and it’s adding some complicated accessibility tests. I’m looking forward to watching it try to fill those out! I also spotted some subtle divergences from the spec that I need to edit. The quiz code I initially generated also included a lot of useful but unasked-for features. They were improvements, for sure, but it was definitely not an MVP. It will be interesting to see how easy it is to work with Q on my more complicated application.