DIY Databases are Coming

“The software revolution has given people access to countless specialized apps, but there’s one fundamental tool that almost all apps use that still remains out of reach of most non-programmers — the database.” on CrunchBase

Database technology is boring but immensely important. If you have ever been working on a spreadsheet and wanted to be able to click on the contents of a cell to get to another table of data (maybe the cell has a person’s name and you want to be able to click it to see their phone #, photo, email, etc), then you’ve wished for a DIY database.

I’ve been waiting for this technology for many years and am happy to report that it’s nearly arrived. Two startups are taking on the DIY database challenge from different sides:

Screenshot from 2016-01-20 18:20:27
If you can make a spreadsheet you can make a map with Awesome-Table.

Awesome-Table is a quick and easy tool for creating visualizations of data inside Google Sheets. It offers a variety of searchable, sortable, filterable views including tables, cards, maps and charts. They’re easy to embed so they are great for creating and embedding directory data onto websites. Here’s an awesome table visualization of
worker coops in NYC.

AirTable is a quick and easy way to create tables that connect to and reference each other. This allows for multi-faceted systems you can travel through by clicking on entities. For example, you can define people in one table, organizations in another, and offices in a third, and then connect them all together so a user can browse a list of people, click on an individual’s organization, and then see all that organization’s information, including its many offices. Pretty useful!

The progress of these two startups leads me to believe we’re less than a year or two away from truly lightweight, easy to use, free of cost, DIY database building systems, and an open source one not too long after that.

The increasing accessibility of database technology has a lot of implications. The most obvious one is that it will enable people to build their own information management systems for common use cases like contact directories, CRM systems and other applications that just can’t be done with existing spreadsheet technology. This will make a wide variety of solutions more accessible to people – so if you want to start or run a business, manage common information resource, or just organize personal information better, you’ll enjoy DIY databases very much.

More interesting to me is the implication that they can have for people trying to reform and democratize institutions.

If you spend time in the type of information management systems used by institutions big and small – whether it’s government agencies like the sanitation department or educational ones like high schools or universities, you’ll quickly notice that many of their most useful and critical tools are nothing more than a set of data tables (directories) and visualizations of the data contained therein (search/filterable tables, cards and maps of that data.)

Screenshot from 2016-01-20 18:19:07
Turn spreadsheets into searchable/filterable directories with Awesome-Table.

These very rudimentary but widely used internal software systems not only define the information people within that institution can access and share, but also limits them to very specific workflows that are implicitly or explicitly defined in the software. Since workflows define the work people actually do, the people who control the workflow are also people who control the workers.

If you want to change how an institution does things, you have to be able to change its information management systems. Since current database technology requires specialized software coding skills, changing these systems often turns into a bureaucratic nightmare filled with bottlenecks. First, a specific group of pre-approved people need to agree to design and fund a change, then another specific group of people need to program and implement the change, and yet another group is often tasked with training and supporting users who then have to use the updated system. That creates a lot of potential bottlenecks: executives who don’t know a change is needed or don’t care enough to fund the work; managers who don’t want to get innovated out of a job or don’t know how to design good software; technologists who don’t have the time to implement a change or don’t have the motivation to do the job right. With all those potential bottlenecks it’s easy to see why so many well funded institutions have such crappy software and archaic workflows.

When people try to improve institutions, they are often trying to improve workflows so more can get done with less time and resources. Unfortunately, the people who actually know what changes need to be made are rarely in a position to control the architecture of the databases they use to get things done.

With DIY databases, people within institutions can circumvent all these bottlenecks simply by making superior systems themselves. This can change a lot more than simply the type of information people have access to – it allows them to explore news ways of being productive.  What they’ll inevitably discover, particularly if they’re in an institution that spends a lot of time managing information, is that they can do a better job managing information than many of their bosses.

DIY databases are enabling the type of horizontal and bottom-up innovation essential not just for better functioning institutions, but also more democratic ones. Databases are the “means of production” for many information workers. When they can build and own their own ones, they’ll be able to achieve more ownership of their own work and take another big step towards being able to manage themselves.

Of course, as technology improves and creating you own databases becomes easy, the hard part will certainly become getting peers to use them.  That’s a topic for another day.

As If We Couldn’t

If you listen to media’s murmuring, you’d think that the American people were ready to give up.  The bankers are too crafty, the corporations too powerful and the politicians too pliant.  We don’t know which way is up, where to turn or on whom to depend.  We’re lost, sad, unhappy, and maybe a little overweight.  For a few brief moments we had HOPE that that America’s most powerful institutions could still solve our problems during commercial breaks.  Yes we can, they whispered in our ears.  Yes we can…

As if we couldn’t. As if the American people couldn’t recognize a lie when it’s told to our face.  As if the American entrepreneur couldn’t maneuver around the obstacles created by international bankers and their confused economists.  As if the men and women in our Armed Forces who committed to defending our Constitution couldn’t stand up to anyone who demanded that they oppress their fellow citizens.  As if our farmers couldn’t team up to develop their own technologies, save their own seeds and reap their own harvests.  As if honest people couldn’t win elections.

“You can fool some people all the time, and all the people some of the time, but you can’t fool all the people all the time.”  I think Bob Dylan said that.  Americans are getting smarter – not the coastal elites who think reading the New York Times qualifies as civic participation – but the “dumb” Americans who’ve enough common sense to recognize that Big Government and Big Business have teamed up to steal our wealth while misdirecting us with the help of the corporate media.  But they don’t control everything on the internet, and as the veil is lifted, both the problems and solutions become much more clear.  I’m particularly interested in the solutions.

While Silicon Valley gets all the press for their trendy products, thousands of programmers, engineers and developers across the world continue to build the free, libre and open source technologies upon which local economies depend.  While multinational chemical companies receive prestigious awards for monopolizing agriculture, American farmers continue to share best practices, organizing seed banks and reach customers directly through farmer’s markets.  While central bankers act like economic gods with their reactionary monetary policies,  local entrepreneurs continue to discover new opportunities to create real value for their communities.

If you’re looking for problems, you’re going to find prestigious people in glossy magazines whining about the difficulties that lie ahead.  If you’re looking for solutions, you’re going to find unrecognized folks working  quietly and feverishly, as if the fate of the world depended on it.  The time for criticism is over.  The time for action is here.

If you’re interested in the emergence of the new, participatory, local-networked economy, what Buckminster Fuller would describe as “a new model that makes the existing models obsolete,” start reading about free/libre/open source technology and ask yourself this simple question: what if everyone had the tools they need to create what they want.

The best has yet to come.