![\"\"](\"http:\/\/www.fogcityjournal.com\/wordpress\/wp-content\/plugins\/2012\/04\/system-failure-matrix-code-wallpaper.jpg\" "\\"system-failure-matrix-code-wallpaper\\"")
<\/p>\n<\/p>\n
By Chris Roberts<\/a><\/strong><\/p>\n April 2, 2012<\/p>\n Technology is never far from Mayor Ed Lee’s mind or lips. With good reason: piles of cash from Silicon Valley luminaries representing Google, Facebook, Salesforce.com and others compelled celebrities like MC Hammer and Brian Wilson to dance for the heretofore obscure former City Administrator<\/a> and propelled Lee, the onetime interim mayor, to election to a full term as San Francisco\u2019s chief executive last November.<\/p>\n Following his lavishly appointed and well-attended victory party at the Palace Hotel, Lee was quick to signal his appreciation for his friends in the technology sector by gifting Twitter, Inc. a generous tax break to persuade the estimated $8 billion company to not leave the city that \u201cgets it done.\u201d<\/p>\n Board President David Chiu.<\/p><\/div>\n Popular tech buzzwords like “entrepreneurship” and “innovation” now spice up Lee\u2019s speech,<\/strong> the latter of which the mayor chose as the subject for his inaugural column for the San Francisco Examiner.<\/p>\n “Innovation is the driver for job creation in San Francisco,” wrote Lee, the mayor of the self-declared “innovation capital of the world,” where 1,500 tech firms employ 30,000 people, just the tip of a technology peninsula that spans from North Point to San Jose.<\/p>\n But there\u2019s a problem for Lee and his image as an innovative tech mayor, one that he inherited from former Mayor Gavin Newsom – the city\u2019s antiquated information technology systems.<\/p>\n “We’re in 2012,” said Board of Supervisors President David Chiu, who sits on the city’s Committee on Information Technology (COIT), a confederation of department heads and bureaucrats tasked with overhauling the city’s IT infrastructure and systems, “but our city’s information technology is stuck in 1999.”<\/p>\n All Gloss and Glamour There\u2019s no doubt San Francisco\u2019s government has embraced technology. \u201cTwitter Prince\u201d Newsom was one of the social media Web site\u2019s very first celebrities, using the-then startup to announce his aborted bid for California governor (so total was Newsom\u2019s devotion to Twitter that he showed off the company\u2019s Folsom Street HQ to Russian President Dmitry Medvedev during the foreign leader\u2019s 2010 visit to San Francisco<\/a>).<\/p>\n Newsom insisted city departments become active on social media. Citizens can tweet to DPW when their driveways are blocked or when trash is not picked up, and can similarly publicly query 311 via Twitter \u2013 and Newsom used his clout to pass some of the nation\u2019s first open data legislation<\/a> (though, years on, city departments are asked, not required, to share data with the public).<\/p>\n Chief Innovation Officer Jay Nath.<\/p><\/div>\n Under Lee, the city created what it says is the nation’s first “chief innovation officer” position, filled by Jay Nath (a position since copied by other cities), and made much of partnerships with the private sector<\/a>, intended to get apps created and mine the private sector for ideas meant to “make city government better, more responsive, efficient and connected.”<\/p>\n It is no small wonder Lee looked to the private sector to upgrade San Francisco city government. If San Francisco is a broadband-equipped new Macbook, its city government is trying to get by with a 486 processor-equipped computer using dialup. That is, if they had 486s or dialup modems twenty-eight years ago, which is when some of the software currently in use by city departments was developed and installed.<\/p>\n \u201cGavin Newsom used to say that he was the Facebook and the Twitter mayor. Ed Lee likes to say he\u2019s the innovation mayor,\u201d said a high-placed city source, speaking candidly on condition of anonymity. \u201cThe fact of the matter is, the focus is on the glossy and glitzy aspect of what tech could be. Meanwhile, our IT infrastructure is a mess, and nobody\u2019s focused on it.\u201d<\/p>\n The Mess<\/strong><\/p>\n The city’s technology infrastructure has long lagged behind the private sector in efficiency, consolidation, and even basic equipment. The San Francisco Police Department made for headline-friendly outrage when it announced that all officers had finally gained access to email – in the summer of 2011. Other reports have detailed the 15-year odyssey of JUSTIS<\/a>, the still-not-complete technology overhaul of the city\u2019s criminal justice departments, a failure that has been blamed for at least one death.<\/p>\n In 2012, San Francisco city employees use human resources and ledger software developed in the mid-1980s, which the public cannot access, to ascertain line-item budget information; a mishmash of data centers, servers, and computers stuffed in closets house data from payroll information to health records to business and building permits (part of which has only in the last month been backed up to the cloud, <\/strong>a basic first step in Silicon Valley firms), and multiple servers house city workers\u2019 email accounts, all of which were identified as in need of upgrades in city Budget Analyst and Board of Supervisors reports five years ago.<\/p>\n The city also overpays for basic software, with multiple departments paying licensing fees to the same software companies. If the city\u2019s software licenses were consolidated, it is estimated the city could save $50 million \u2013 a good chunk of its current $165 million budget deficit.<\/p>\n This makes conducting simple city business more challenging and more expensive,<\/strong> but it also makes city government less open.<\/p>\n One of Lee’s challengers in the November 2011 election, venture capitalist Joanna Rees, tried to make issue of bloated city salaries and pension benefits on the campaign trail. She requested the Mayor’s Office supply line-item budget information, detailing employees’ salaries down to the dollar (as would be available from any of the technology startups Lee’s government swoons over) but she was told no such information was available, not with the way the city’s antiquated human resources system – called FAMIS – stores data.<\/p>\n No One In Charge<\/strong><\/p>\n Acting Chief Information Officer Jon Walton.<\/p><\/div>\n Lee shares at least part of the blame for the city\u2019s state of IT affairs: the city\u2019s Chief Information Officer reports to the City Administrator, Lee\u2019s old job. Former Chief Information Officer Chris Vein oversaw the release of report after report from the City Controller to the Budget Analyst detailing municipal IT\u2019s shortcomings before he left for a job in government in Washington, DC. But in a way, neither Lee nor Vein, nor Acting Chief Information Officer Jon Walton are to blame \u2013 because no one person is in charge.<\/p>\n This is common among cities, according to Walton. \u201cLarge cities are faced with the common challenge of balancing centralized IT services with decentralized department IT services,\u201d he told Fog City Journal<\/em>. This means Walton must often rely on departments\u2019 individual goodwill to implement reforms or practices recommended, and accountability for deadlines missed or budgets blown is hard, if not impossible to track. Walton chairs a committee called COIT (Committee on Information Technology), which last year issued the city\u2019s first unified citywide IT plan.<\/a><\/p>\n In the Five Year Technology plan released last year, COIT identified ten possible tech upgrades, including a citywide security plan, a citywide dark fiber network, and replacing the telephone system with VOIP (voice over internet protocol) technology. Some of the upgrades mentioned, like a citywide enterprise software agreement \u2013 i.e., one license for word processing and data programs instead of multiple licenses \u2013 were highlighted in the 2007 report<\/a>, and have yet to be implemented (translation from tech-speak: five years on, and the city\u2019s still wasting money on multiple licenses for software).<\/p>\n Three major technology upgrades \u2013 merging the city\u2019s email platforms, replacing the decades-old HR software, and consolidating and safeguarding the city\u2019s data \u2013 are underway. Each has been tasked to a different city department: the Department of Technology, the Controller, and chief innovation officer Nath, respectively.<\/p>\n Email<\/strong><\/p>\n It\u2019s hard in 2012 to get excited over something as basic as email, yet it\u2019s email consolidation that will be one of the city\u2019s biggest IT accomplishments this year.<\/p>\n Years ago, multiple city reports addressed the need for a consolidated email platform, with some city users on LotusNotes, and others on Microsoft Exchange. Today, \u201cI still cannot easily e-mail a city employee,\u201d said Chiu, who named seven different city email servers.<\/p>\n This is at last changing. The city\u2019s email consolidation project is \u201cwell underway,\u201d Walton wrote in an email, with about 1,000 city users on a consolidated Microsoft Exchange platform. That leaves about 20,000 users left to go, however, according to the most recent information provided by Walton in response to an email request. Some city departments have yet to be scheduled for a \u201cgo-live date,\u201d and no users were able to import their contacts, attachments, or calendaring, according to a city report<\/a>.<\/p>\n E-Merge<\/strong><\/p>\n Close to one-third of San Francisico\u2019s $6.8 billion budget goes to city employees<\/a>. The software used to manage the city\u2019s 22,000 employees is 15 years old, a spring chicken compared to the software used to watch over the city payroll, which is 28 years old, according to the Controller\u2019s Office, which is tasked with the $45 million consolidation\/replacement of the ancient software, an undertaking dubbed \u201cE-Merge.\u201d<\/p>\n Deputy Controller Monique Zmuda.<\/p><\/div>\n Replacing the vintage computer systems has thus far taken four years, according to Deputy Controller Monique Zmuda, in part because of software customization by city programmers. The new system will manage \u201call human resources functions, payroll, benefits administration, and time reporting,\u201d Zmuda said.<\/p>\n Still to be done is the replacement of FAMIS, the city\u2019s financial accounting system that is entering its 25th<\/sup> year of service. The cost of this project is unknown as well as the total scope, in part because \u201cmany departments utilize their own small sub-systems to compliment the city’s central accounting system,\u201d Zmuda wrote.<\/p>\n Solving that lack of centralization and putting the city\u2019s data under fewer roofs has been tasked to Nath.<\/p>\n There\u2019s not much room for public visibility of city data, yet. The Controller plans to make available to the public more budgetary data, but to do so in a way that\u2019s \u201cunderstandable and readable\u201d is tricky, Zmuda said. \u201cThe trick is to modify the old system to enable the data to be reformatted and re-sorted.\u201d<\/p>\n Efforts to put the city\u2019s budgeting data in a publicly-available portal were initially dismissed because it would be \u201cdifficult,\u201d according to a city source. Difficult, and more under the purview of the city\u2019s data initiative, which is asking \u2013 politely \u2013 for city departments to make data available to the public on a Web site.<\/p>\n Data<\/strong><\/p>\n Lee and Nath announced in early March<\/a> that the city had moved some of its data from local servers and hard drives to the cloud. The city\u2019s open data is also the envy of the rest of the world, Nath told Fog City Journal<\/em>. With \u201c200 datasets\u201d online, and over \u201c60 apps\u201d developed, \u201cwe’ve been an international leader on open data,\u201d Nath wrote via e-mail.<\/p>\n Yet private companies would still find much lacking in the way the city houses its data, in a m\u00e9lange of data centers, data closets and stand-alone servers. A corporation or another comparable organization should have \u201ctwo, at most maybe three\u201d data centers, according to a high-placed city source. \u201cInstead, we have three dozen.\u201d<\/p>\n San Francisco is budgeted to spend $7 million over the next two years to shore up this situation, with \u201c900+\u201d data locations consolidated into two servers at 200 Paul Avenue, a privately-owned data center in the city\u2019s Bayview District, according to Walton<\/a>, with another city data center located at the airport. The Bayview data center is scheduled to be built by November, with city departments to move their IT servers to the data center in about a year from now, according to documents provided by Walton.<\/p>\n The city\u2019s data is still susceptible to a major disaster. In the event of a major earthquake, for example, city workers would be unlikely to have access to email \u2013 and there are no plans to consolidate the city\u2019s data into a user-friendly \u201cdata warehouse\u201d akin to what was done at the federal level with Data.Gov, according to Nath. \u201cIt\u2019s proved expensive [for other cities],\u201d he said, \u201cand doesn\u2019t really bring us to where we need to be.\u201d<\/p>\n![\"\"](\"http:\/\/www.fogcityjournal.com\/wordpress\/wp-content\/plugins\/2012\/04\/david_chiu1.jpg\" "\\"david_chiu\\"")
\n<\/strong><\/p>\n![\"\"](\"http:\/\/www.fogcityjournal.com\/wordpress\/wp-content\/plugins\/2012\/04\/Jay-Nath1.jpg\" "\\"Jay-Nath\\"")
![\"\"](\"http:\/\/www.fogcityjournal.com\/wordpress\/wp-content\/plugins\/2012\/04\/zps-JW_Picture_5_18_11.jpg\" "\\"zps-JW_Picture_5_18_11\\"")
![\"\"](\"http:\/\/www.fogcityjournal.com\/wordpress\/wp-content\/plugins\/2012\/04\/monique_zmuda.jpg\" "\\"monique_zmuda\\"")