The Open311 specification and implementations have developed rapidly in recent months and several developers and service providers have already integrated the first version of the specification into their technology. San Francisco has also moved their API from just  being available for testing to being deployed as a live, in-production web service. A full list of services and applications using this version of the API will be made available shortly.

From the outset, the API development has been versioned so that new specifications can be advanced while coexisting with older ones. With this in mind, the developer community, cities, and companies have been moving the conversation forward to evolve and improve the first specification so that it can be used by a wider audience.  The mailing list has come alive with feedback on a wide variety of issues and the two cities currently driving this forward the most, San Francisco and Washington D.C., have come together to agree on and fully interoperate with a new Open311 GeoReport v2 specification. Both of these cities are accepting requests for API keys (San Francisco, Washington D.C.) to develop with the new specification. A full run down of the changes and advancements will be made available shortly, but one of the first things you might notice is that we are now referring to this specification as the Open311 GeoReport API. The reason for this name is to clarify the specific functionality provided within the umbrella of all things that 311 services provide. It’s also meant to make it more obvious that the specification applies to those who do not associate these services with the “311″ name or shortcode.

We look forward to hearing your feedback so that we can better clarify the Open311 GeoReport v2 specification and give everyone a better sense of the current state of the Open311 effort. Please speak up here in the comments or on the mailing list. Also feel free to contact me directly. We hope to hear from anyone interested in innovating around this common foundation to help improve our cities and all communities.

Here’s the announcement from SFgov.org:

We are pleased to announce the release of our Open311 API. The design is a result of a collaborative effort between cities, non-profits and developers. We look forward to seeing what you produce.

You can request an API key from their website and read-up on the Open311 API spec and developer resources here on this website. If you have any questions or comments, please leave them here or on the mailing list. We look forward to working with the developer community and seeing some amazing Open311 apps!

As White House CIO Vivek Kundra joined San Francisco Mayor Gavin Newsom, CIO Chris Vein, and Tim O’Reilly to signal the launch of the Open311 API in San Francisco the work towards spreading an open standard gained huge momentum. Yet we still need continued coordination and collaboration to deliver the vision of a distributed open platform that can be used in communities everywhere.

Both San Francisco and Washington D.C. are preparing to deploy their Open311 APIs within the next week and we will continue to provide updated information on API methods, registering API keys, and the distinction between using the API as a sandbox for testing and using it for full deployment. We will also be sure to clarify any nuances regarding the API implementation in these different cities - such as XML versus JSON output.

This is also a good opportunity to provide more background on the development of Open311. First, the history of the open model:

FixMyStreet was launched in the UK by MySociety almost exactly 3 years ago. This was followed by similar projects in other countries like Verbeter De Buurt in The Netherlands and SeeClickFix in the U.S.. About a year ago John Geraci started a dialog on DIYCity about providing this capability for the 311 system in NYC. This conversation caught the attention of The Open Planning Project which had been developing the idea too. The conversation also caught the attention of Dmitry Kachaev in Washington D.C. who was preparing to launch the second round of the Apps for Democracy contest. As D.C. prepared to open their 311 API, The Open Planning Project began to coordinate this as something that could be made into an open standard and created Open311.org to facilitate this.

Working towards an open standard: In mid 2009 D.C. developed their initial 311 API spec and SeeClickFix also released a draft spec. D.C. launched their API in the summer of 2009 and had many apps developed around it as part of their Apps for Democracy contest. In October of 2009, we held the Open311 DevCamp to help coordinate the development of this standard and related technologies with many different cities and companies. This also signaled the initial development of the Open311 API spec in San Francisco. The team in San Francisco led by Alissa Black and Jay Nath continued to facilitate development of the spec over several months with the input of many people involved in this effort like Dmitry in D.C., Kam Lasater at SeeClickFix, and myself. The current spec in San Francisco reflects this collaboration and their leadership to convene others. The D.C. API spec has also been undergoing a revision to bring it more inline with the spec that has been developed in San Francisco. The specification will continue to evolve through incremental iterations to allow greater interoperability and a better experience for developers and citizens using Open311 services.

The Open311 API spec coordinated by San Francisco is available on the wiki. Despite the fact that the APIs have not quite yet been deployed and some details of the spec have yet to be fully documented, we can already see some examples of the developer community preparing to work with it.

We look forward to working with other cities in support of this standard. Some cities like Boston and Edmonton are set to open their APIs in the near future and other cities like Seattle and Portland do not yet have a call center and should stand to gain a lot by being involved with this initiative from a fresh start. We also look forward to closer involvement with international efforts like Ushahidi. Ushahidi is a collaborative issue tracker which played a crucial role in the response to the earthquake in Haiti.

Stay tuned for more details about the deployment of the APIs, the release of reference implementations, and code libraries that can help provide integration with existing 311 systems.

Please leave us with your questions and comments and let’s continue to work together.

As San Francisco and Washington D.C. prepare to launch their new APIs we’d like to have other cities and managers of 311 services show their support for implementing an interoperable standard for these APIs. The more cities get behind this effort, the better for developers, for city budgets, and for citizens. Showing a critical mass of support for using this standard will help encourage developers, other cities, and existing 311 services to become more invested in the potential of an interoperable system.

A standard means that people can use their favorite app in every city and developers can focus on new features rather than different requirements for each city. Everyone can then benefit from new innovations built from a common foundation. Additionally, interoperability is helpful not only to simplify the offering of citizen interfaces, but also to help unify and better facilitate inter-agency coordination for city managers.

To date, the people who have joined San Francisco and Washington D.C. by showing their support for implementing a standard API include directors of these services and CIO or CTOs in cities like Boston, Portland, Edmonton, Seattle, and Los Angeles as well as those from services like SeeClickFix and Ushahidi.

Please take the pledge and show your support.

Both San Francisco and Washington D.C.are preparing to release their Open311 APIs next month.

Over the past several months, Alissa Black and Jay Nath  having been working within the technology team in San Francisco to incorporate public feedback for the development of the Open311 API. The current proposal can be seen on the wiki.

Dmitry Kachaev in Washington D.C. is also preparing to relaunch the D.C. API incorporating the changes of their v2 draft while paying attention to the API spec that San Francisco has worked on in an attempt to start working out interoperability between cities.

Progress has also been made on ways to provide integration between geographically disparate APIs. The current proposal is an approach modeled after DNS and reverse-geocoding called GeoWeb DNS. The way this fits in with the Open311 API is illustrated in this system architecture diagram. There’s a description of the GeoWeb DNS service on the wiki and a reference implementation is available at: http://geodns.open311.org

The San Francisco team is holding their final public conference call before they launch their Open311 API. The call is open for all to join and provide feedback, comments, and ideas as they move from their development phase to the actual launch. The invitation for the call as posted on the sfgov.org website:

Public Conf Call 2/25 @ 10AM PST

Please join our public conference call this Thursday @ 10AM Pacific.

Dial in: 641.715.3625
Pin: 813951#

The topics to be discussed are:

• design spec
• other cities pledge to use standard Open311 protocol
• feedback on a few modifications we may make to spec (addition of metadata call)
• open questions
• mailing list Please join!

Please join the conversation and help ensure that the APIs are well equipped to serve these cities and the developers working with them.

The earthquake in Haiti can be an important reminder of role that communications technology plays in dealing with crises. I also think this is an opportunity to consider how our use of day to day technologies can better prepare us for responding intelligently when there is an emergency. As described by this Urban Omnibus story about the seemingly benign NYC steam system, there are cases where 311-like reporting has the potential to prevent disasters in the first place.

Within the software development field, there is rarely a distinction between the tool you would use to report a problem and the tool used to make a request for an enhancement. Additionally, there is rarely a distinction between the tool you would use for various severities of problems be it a minor user interface bug, a bug that causes a serious error, or even a bug that exposes a major security hole. For example, if I was going to report a new bug about the Firefox web browser, my options would look like this:

Since bug reports are usually made public I can first see if the issue has already been reported and add to the original report instead. This same open source philosophy is what underpins the concept of an open 311 and we were very conscious of this in adapting the Trac bug tracker into the GeoTrac issue tracker.

In this context, the difference between 311 and reporting software bugs is that 311 is explicitly meant to deal with non-emergencies. However, I am now beginning to wonder if this should always be the case. Emergency services stand to benefit from being interoperable with the ecosystem of 311 technology. That said, there are many good reasons why services like 911 and 311 are separately prioritized systems and I certainly don’t want to encourage 311 requests to go to 911 or vice versa. I do however advocate for 911 to integrate with and provide the same technologies that 311 systems offer. A person should be able to report on and follow-up on an issue the same way regardless of whether it’s a crime, an accident, or a non-emergency. Both 311 and 911 phone calls could even be followed by an SMS message containing a link or SMS script to allow you to update the issue with a precise location, photos, videos, and more information. Even though 311 and 911 are meant to serve different priorities, they have many of the same requirements, so it makes sense to utilize parts of the same platform for both contexts. Yet there are other reasons why emergency and non-emergency services might benefit from sharing the same platform.

In a major crisis there is some chance that communications for emergency services will not be able to serve the demand - either because of heavy call volume or because of compromised communications infrastructure. As reported in this Wall Street Journal story, the communications system in Haiti was not fully holding-up following the earthquake. Additionally, there are many crises, such as widespread disaster or politically fueled turmoil, where the authority responsible for emergency services has been compromised. In situations like these, you have to make use of self-serviced issue reporting, SMS if possible, and if all else fails - some kind of adhoc-communication system. When a cellphone network is under heavy load or has intermittent connectivity, SMS is a much more reliable way to communicate than voice phone calls. These types of environments are exactly what crowdsourced crisis mapping tools like Ushahidi were created for.

Ushahidi is the crisis mapping tool seen in use at the top of this post. It’s an open source platform that was originally created to receive SMS and web-based reports of violence and other issues during the fallout from the 2008 elections in Kenya. Shortly following the Haiti earthquake an instance of Ushahidi was made available to start collecting reports from the field. Reports can be sent via SMS, email, twitter or a web form. More details are available at http://haiti.ushahidi.com. This may look and sound familiar. On a technology level, there is very little differentiating this from a 311 issue-reporting system. And sure-enough Ushahidi is in use for more 311-like applications such as Atlanta Crime Reports shown here. Compared to traditional emergency communications systems, there is one major disadvantage to setting up an adhoc system like Ushahidi: familiarity. Since this instance of Ushahidi only came into existence following the earthquake, its use is probably stymied by lack of awareness more than anything else. Ideally, the word will spread, communications systems will stabilize, and it will offer a core utility for the process of recovering from the disaster. Furthermore, the tool will hopefully stay in place permanently and more people will become aware of it and relevant authorities will learn how to work with it. The next time an issue comes up people won’t have to learn about something new within a potentially chaotic and communications-troubled environment.

People in the U.S. and Canada have been familiar with 911 as a standard way to ask for help for about 40 years and many cities have become very familiar with 311 in more recent years. New Yorkers are incredibly familiar with the utility that 311 offers them. We are also constantly reminded, “See Something? Say Something,” (though surprisingly the posters that announce this instruct us to call neither 311 nor 911). It turns out that New Yorkers are so familiar with 311 that by 11am Wednesday morning the NYC 311 system had received over 10,000 calls regarding the crisis in Haiti. Teaching people how to use a system to deal with small problems on a daily basis is a good way to prepare them for using the system to deal with an emergency.

The challenge then is to provide a platform that people can be familiar with, that is accessible through a variety of means, and that routes information to the right place without fragmentation or duplication. This is the goal of the Open311 effort and it’s the goal of those working on crisis mapping technologies. Where ever I am I should be able to do something as intuitive as sending an SMS to 311 or using my favorite 311-like app and know that my report is being routed to the correct agency. This kind of infrastructure should be available everywhere so that it’s ready for a crisis. Like the internet it should be a robust, distributed, fault-tolerant system so that it can withstand a major catastrophe like an earthquake.

I hope to coordinate with the those developing emergency services and crisis mapping so that the technology built for the benefit of 311 can also be used in a time of emergency. There is clearly an opportunity to integrate or complement existing standards for emergency management (like the Common Alerting Protocol and IPAWS) with the emerging standards around 311. Additionally, Ushahidi is not alone in this space. Ushahidi uses FrontLine SMS for text messaging and there are other SMS platforms like RapidSMS and Slingshot SMS as well as tools like GeoChat and SwiftApp that contribute to this field. Using the best qualities of tools like Ushahidi with the increasing familiarity and ubiquity of 311 applications, we can provide a healthier and more interconnected immune system to help communities in need.

San Francisco is the city currently leading the charge in the Open311 effort and they are holding a public conference call to solicit feedback on the development of their API and I will quickly follow that up with updates here about their progress as well as new developments like GeoWeb DNS.

Please help support the enduring spirit of the people of Haiti and help build a better future for the poorest nation in the Western hemisphere: http://www.google.com/relief/haitiearthquake/

This follow-up is long overdue, but progress has certainly not slowed. My initial DevCamp follow-up pointed to most of the resources that came out of the event, but I’ll run through them again. The video stream of the DevCamp is available here. The IRC log is available here. Some of the main wiki pages created are:

• Documentation of existing 311 systems in cities (Cities, please edit these. Thanks for adding yours Toronto!)
• Creating a universal Open311 API
• Pitching cities on Open311
• Usage and analysis ideas for 311 data

So again, the main focus is still coordinating a standard API, but this group should also be leveraged for sharing best practices around 311 services in general. Please use the wiki and mailing list to help facilitate better communication between cities and other entities. If you are not already on the mailing list, please subscribe at: http://lists.open311.org/discuss

For a more full follow-up, let me review some of the things that left the strongest impression on me both during and following the DevCamp:

The need for a two-tiered model for submitting service requests

This issue comes up over and over again when one looks at the usability of service request systems currently in place in most cities. The problem is that most existing systems have a very cumbersome process of collecting a required set of fields in order to submit a request. The problem isn’t as apparent when speaking to a call center because the complexity of required fields is often completely curtailed by the operator, but with most on-screen interfaces the complexity of the required fields often makes the system unusable. This was one of the main topics of conversation when discussing a standard API and we couldn’t help but joke about the absurdity of some of the required fields for some service-request types in DC’s 311 API, like the list of available colors to identify an abandoned bike. Dmitry conceded that if an image is attached to a submission it can negate the majority of required fields for many service request types. In some cases, the problem is just identifying the service request type from the beginning. For example, try to navigate the ontology to report a pothole in NYC’s 311 list of request types. So the problem remains, how do you provide a user interface that makes it extremeley easy for someone to submit a request without too many required fields or a complex ontology of service request types? I think the answer is a two-tiered approach. First you allow submissions with the most basic core set of fields. The submissions then sit in a public holding bay where you or anyone else, including a city worker, can review the submission and help complete all the required fields. This two-tiered approach is the process we are currently using for our FixCity Bike Racks project.

The value of improved inter-agency communications

When I talk about the value of opening and standardizing 311 systems, I tend to look at it from outside government and focus on the benefits of better connecting citizens and government. However, many of the people at the DevCamp from within city governments mentioned the value of improving inter-agency communications and I think that’s really important to keep it mind, so I’ll be sure to emphasize that in the future.

The cost of SMS and the value of voice

Something I tried to say whenever discussion of Twitter integration came up was that if you’re discussing short-from submissions, then you should also be talking about SMS integration. Since SMS provides much wider coverage to a much wider demographic than Twitter, I thought this was an important accessibility (and digital-divide) issue to raise. Having said that, I was reminded of how expensive SMS is for services to support (and there’s some discussion of this on the TOPP Labs blog today). The far more universal and far cheaper medium is voice (phone calls) and obviously that is how the vast majority of 311 communications occur. As I mentioned in my Gov 2.0 talk, we are now at a point where voice can seamlessly integrate with web APIs. I don’t think this is something that received nearly enough attention at the DevCamp. It turns out that on the day before the event, Mark Headd wrote an excellent post about voice and web APIs for 311 on the Vox Populi blog. I know Troy Davis from CloudVox participated in the DevCamp remotely, so maybe he has more to say on this as well. Ribbit just opened up their voice/web-api service and there are already some cool demos starting to come out of that.

The layers of the Open311 model:

This rough outline is the result of further considering how the different parts of the Open311 model must to fit together

• End user
• Client application
• Open311 API - decentralized read/write web APIs. 2 step submission:
  • Simple and easily accessible (minimum fields required, status: “submitted”)
  • Crowdsourced validation (city requirements, status: “validated”)
• Adapter to translate existing systems for Open311 web APIs
• Existing CRM/311 system
• Government

Open source implementations

There are at least three open source service request web apps that are available for anyone to implement a draft API spec.

• FixMyStreet (source code)
• FixMyStreet.ca (Django source code)
• GeoTrac (installation and source code)

Developing the Open311 Spec and API

An abstract and some of the terminology relating to an API was put into the wiki on the Open311.org Draft Spec page and I’ve started sketching out my interpretation of a possible schema in JSON on that discussion page. Syntax highlighting is enabled on the wiki if people want to put in snippets of code or structured data to discuss.

We should continue to help coordinate the details of the spec, but I think this should be done simultaneously with iterative  reference implementations. I think slight variations of APIs and schema like this will emerge both experimentally and in-production. San Francisco’s 311 API will be a useful reference point since it will represent the second city (after DC) to provide a read/write API and the first real opportunity to consider interoperability. With continued coordination we can see what works best.

Action Items

1. Get everyone on the mailing list
2. Continued coordination among those working directly with API implementations.
3. Release reference implementations, client apps, and web API adapters for existing systems

So there’s a bit of a brain dump for now. I look forward to hearing your feedback and continued coordination on the mailing list and wiki.

We’re still in the process of collecting all of the material from the DevCamp, structuring it on the wiki, and recharging the mailing list to coordinate next steps for developing the API. For now, this video provides a good summary of some of the things that were covered at the event. The video is broken into groups which have wiki pages to organize their further development:

1. 311 usage and analysis brainstorm: Video bookmark / Wiki page
2. Documentation and resources to explain the value of open 311 systems: Video bookmark / Wiki page
3. Technical spec working group, creating a standard API: Video bookmark / Wiki page
4. Final closing remarks: Video bookmark

The wiki was also seeded with other new content including descriptions of the 311 systems in the cities that people were representing. We’ll be sure to follow up with the rest of the video content, a full summary blog post, and further coordination on the mailing list.

On October 24th, The Open Planning Project will host Open311 DevCamp at their NYC office. Please register to attend either in-person or remotely via Eventbrite (it’s free). This is a DevCamp style un-conference to coordinate a standard specification for 311 services. Washington D.C’s 311 API will be a major case-study for developing a more universal 311 API. In general, this DevCamp will be an opportunity to discuss and develop what’s needed to make 311 services more accessible and for cities to share knowledge for mutual benefit. The event is intended for developers, project managers, and policy makers involved with 311 services. We encourage those involved with 311 services from all cities to take part. If you cannot attend in person, please sign up as a remote attendee and we’ll provide you with information about how to connect to the DevCamp remotely.

Please register at http://open311.eventbrite.com
The wiki page for the event is http://wiki.open311.org/Open311DevCamp

Project 10^100 - Create real-world issue reporting system

Last year Google created Project 10^100 as a call for ideas to change the world by helping as many people as possible. It took them a while to review all of the submissions because they received about 150,000 spanning 172 countries. A number of the submissions described similar ideas, so Google narrowed them down to 16 core ideas. Two of these ideas describe what we’re doing here with Open311. The more broad idea is to Collect and Organize the World’s Urban Data which includes the example of “creating a website where residents can text or upload photos that highlight city issues,” but the idea that is truly spot on is to create a real-world issue reporting system. Please vote for these ideas (before October 8th!) and let’s use Open311 as an organization to bring this to life.

Google’s Project 10^100: Create real-world issue reporting system

Build an issue-reporting website that lets people report problems to proper authorities. When software testers find errors, they generally submit them to a tool which automatically routes them to the right team to be fixed. Implementing proposals for this idea might involve creating an analogous system for the real world that lets anyone report a problem of any kind (e.g., a dangerous pothole), and routes problems it deems sufficiently important to the proper authorities (e.g., the relevant road agency). The aim would be to incorporate all the niche applications that users suggested, including reporting crimes to the police and environmental issues to local governments.