Where good ideas come from (for software engineers)
Stop waiting for a eureka moment. Innovation is a process, not magic. Learn to optimize your environment, tinker, and collaborate to solve complex problems.
Get the free AI Agent Building Blocks ebook when you subscribe:
I used to think solving problems came from locking myself in a room and staring at code until I had an idea. I was wrong.
There isn’t a magical “eureka moment” that solves a complex problem. Instead, we could create a productivity system to solve these problems, deterministically.
History shows that innovation is rarely a flash of individual brilliance. It’s a process that relies on specific environmental conditions. We can optimize for that environment
I broke down Steven Johnson’s book "Where Good Ideas Come From” and distilled the actionable insights from it. I decided to keep a lot of examples from the book to see how the ideas apply in other areas than software engineering.
Let’s expand your adjacent possible! (Keep reading and this last point will make sense 😉)
In this post, you’ll learn
Why the “lone genius” is a myth and how great ideas actually happen.
How to tinker with existing tools to build the future.
How to build on top of platforms to accelerate your output.
The importance of liquid networks and sharing your work early.
The Adjacent Possible: Tinker more
Innovation isn’t a huge leap into the unknown. There are no research papers without a bibliography; nobody invents something completely new. They always cite others’ work. Innovation is more about opening specific doors that are available from the room you are currently standing in. For example, simple molecules had to combine into cells before the door to complex organisms could open. You cannot jump from a single-celled organism to a human in one step.
Great engineers are tinkerers. They don’t invent sci-fi technology from scratch. They put together existing APIs and libraries to solve new problems. Even with AI, researchers did not invent the transformer in a vacuum. They tinkered with existing text and image generators like GANs until they created a new spare part. That new part allowed companies to build ChatGPT and Claude.
I think this concept relieves a lot of pressure from all of us. We don’t have to be some genius inventing science fiction tech. Also, if there is some technology that looks like science fiction for us, it means we have to keep learning and expand our adjacent possible to be closer to the state of the art.
There is a trap in trying to skip steps in the adjacent possible. Charles Babbage designed a programmable computer in the 1830s using steam and gears. It was a brilliant idea that failed completely. The vacuum tubes were not in his adjacent possible. He was missing that spare part to combine everything into a computer. In contrast, YouTube succeeded in 2005 not because the idea was new, but because the ecosystem of Flash, broadband, and cheap cameras was finally ready.
The actionable advice here is to focus on first-order combinations. You should look at the tools currently in your stack and ask what you can build today. Do not try to be ahead of your time. Try to be exactly on the edge of the present possibilities.
Also, this is a reminder to avoid comparing your level 1 with someone’s level 100; their adjacent possible is outside your league.
Liquid Networks: Collaborate
The book uses gas, liquid, and solid states to explain collaboration:
Gas is too unstable, like a group of people talking different languages and unable to communicate.
Solid is too rigid, like a big corporate hierarchy.
Liquid allows for a lot of connections with enough plasticity to change.
Kevin Dunbar conducted a study to see where scientific breakthroughs actually happened. He found they rarely occurred when a scientist was looking through a microscope alone in the lab. The breakthroughs happened at the weekly lab meeting. This was a liquid network where scientists discussed.
You can apply this by treating code review as your conference table. Don’t review PRs as a chore. View them as a collision point for ideas. This also applies when you review an architecture proposal or any other document. Participate in internal chats or open discussions. Your ideas alone aren’t valuable, and you’ll make progress in the idea by soundboarding it against different people.