Who can you trust? IARPA Challenge Puts Trust to the Test


Who can you trust? The answers to that question can be both difficult and essential for society in general, but particularly vital for the intelligence community. So what if an algorithm existed that could identify neural, psychological, physiological and behavioral signals to determine a person’s trustworthiness? Thanks to a new competition from the Intelligence Advanced Research Projects Activity (IARPA), that could be possible.

The INSTINCT Challenge, which began February 19 and runs through May 5, is the first such event from IARPA, sponsored in partnership with the Air Force Research Laboratory. It seeks to study neurophysiological data as it correlates to trust. Participants must deliver source code and written documentation implementing an algorithm to detect, measure and validate one’s own signals to assess another person’s trustworthiness more accurately. A real-time online scoring utility and leaderboard will track algorithm performance throughout the challenge.

Three prizes will be awarded for the programs that best predict trustworthy behavior: $25,000 for first place, $15,000 for second and $10,000 for third. Visit the challenge website for full details.


$1 million cash prize for drone innovations


(Muaz Shabandri) / 11 February 2014

The UAE government announced a new international award worth $1 million for promoting innovation in drone development aimed at helping governments serve people better. Mohammed bin Abdullah Al Gergawi, UAE Minister of Cabinet Affairs announced the new award on Tuesday at a special media briefing. 

‘We invite individuals, businesses and universities to work on their best ideas and look at innovative ways of improving the existing technology for drones. UAE wants to develop drone technology and help reduce the time taken for providing government services,’ he said.

A team of experts working closely under the supervision of the UAE Prime Minister’s Office has already tested the possibility of using unmanned aerial vehicles to provide government services. In three months, pilot tests and proposals have been evaluated for implementing this technology in the UAE.

Photo by Juidin Bernarrd/Khaleej Times

‘The world is becoming more technologically advanced and we believe in smart governance. It isn’t simply about using technology but it also includes looking at ways to improve government offerings,” said Gergawi.

Universities, businesses and individuals can participate in the award, which will be open in two categories – international and UAE. Innovators in the UAE can aim for a top prize of Dh1 million, while the international category will have a prize of $1million.

In a video demonstration of the current drones tested by the UAE government, a package carrying an Emirates ID card can be delivered from the ID center to a resident’s house.

‘Legislations are extremely important when it comes to using drones and we need to work with Civil Aviation Authority and other authorities before implementing these new technologies,” added Gergawi.

A jury of specialists and government officials will evaluate the entries and technical guidelines for the competition will be unveiled in two weeks. Entries will have to meet global safety standards and innovators will be challenged to create drones which can cover reasonable distances and remain reliable.

Contestants will also be expected to submit detailed proposals with the technical features and how it could help improve an existing government service.

Winners of the international competition will be announced at next year’s government summit in February 2015. The competition’s UAE edition would come to a close in May 2014.


For more news from Khaleej Times, follow us on Facebook at facebook.com/khaleejtimes, and on Twitter at @khaleejtimes

GSA’s Challenge.gov Wins Harvard Innovation Award


Online portal features information on government challenges and competitions, aims to crowdsource creative ideas to solve societal problems.

The General Services Administration’s (GSA)Challenge.gov online portal for running challenge and prize competitions has won Harvard University’s Innovations in American Government Award. GSA was chosen as a front-runner of the prestigious award from a pool of more than 600 applicants.

Challenge.gov was launched in July 2010 in response to a memo by the Office of Management and Budget (OMB) instructing the GSA to build a platform that allows the public to compete for prizes by “providing the government with novel solutions to tough problems,” the GSA said. Challenge.gov uses a pay-for-performance model and allows the government to crowdsource such solutions in a contest format. The contest site is powered by a platform called ChallengePost.

Between Sept. 2010 and Sept. 2013, 58 federal agencies used Challenge.gov to run 288 competitions, according to the GSA. The competitions encompass everything from science to engineering to design, resulting in many cases in public-private partnerships. “Prizes and challenges are being integrated into the fabric of the way we do business in government,” White House Office of Science and Technology Policy (OSTP) director John Holdren said at an event honoring GSA on Jan. 23.

Some examples of contests on Challenge.gov include Federal Trade Commission’s (FTC) Robocall Challenge aimed at blocking auto-dialing telemarketers. FTC picked two winners, each of whom received $25,000 for their proposals for technology to intercept and filter out robocalls. The Labor Department’s Office of Disability Employment Policy (ODEP) launched the Disability Employment App Challenge, calling on entrepreneurs and developers to use ODEP’s publicly available data to create apps that assist disabled job seekers. The Department of Agriculture (USDA) and Michelle Obama used Challenge.gov to create the Apps for Healthy Kids Challenge, which focuses on engaging software tools and games for children.


Challenges, prizes to play larger roles in agency efforts


Challenges, contests and prizes will become a bigger part of how agencies develop solutions to their problems, according to federal officials.

GSA announced Jan. 23 it was selected as the winner of the Innovations in American Government Award by the Ash Center for Democratic Governance and Innovation at Harvard University for its work developing the contest and award platform Challenge.gov.

John Holdren, the Director of the White House Office of Science and Technology Policy, said at the event challenges allow agencies to pay only for results outside of the traditional procurement process.

“Prizes and challenges are being integrated into the fabric of how we do business in the government,” Holdren said. Agencies across government are crafting additional guidance to help spur the use of challenges, he said.

“Agencies need to keep seeking new ways to use prizes as tools to advance their missions and help solve the great challenges facing societies today,” Holdren said.

Agencies have used Challenge.gov to support more than 300 contests and have drawn 42,000 participants and 3.6 million visits to the website, according to GSA.

Some of the challenges included:

■ A 2009 award by NASA to anyone who could develop a more flexible astronaut glove capable of performing a wide array of tasks in the vacuum of space.

■ A contest to see who can build the best app to help track personal financial issues that led to the development of an app to track and pay for student loans.

■ An ongoing challenge from the Health and Human Services Department to develop a model to predict future influenza outbreaks based on publicly available data.

Dan Tangherlini, GSA administrator, said he sees the first 300 challenges as a baseline for future efforts and evidence for agencies that they can use Challenge.gov to help solve a broad range of problems.

“It’s a fantastic way of really changing the way we go to market and ask for problems to be solved,” Tangherlini said.

DARPA ANNOUNCES CYBER GRAND CHALLENGE #publicprize #openinnovation


First-of-its-kind cyber defense tournament seeks to drive automation revolution in information security 

What if computers had a “check engine” light that could indicate new, novel security problems? What if computers could go one step further and heal security problems before they happen?

To find out, the Defense Advanced Research Projects Agency (DARPA) intends to hold the Cyber Grand Challenge (CGC)—the first-ever tournament for fully automatic network defense systems. DARPA envisions teams creating automated systems that would compete against each other to evaluate software, test for vulnerabilities, generate security patches and apply them to protected computers on a network. To succeed, competitors must bridge the expert gap between security software and cutting-edge program analysis research. The winning team would receive a cash prize of $2 million.

“DARPA’s series of vehicle Grand Challenges were the dawn of the self-driving car revolution,” said Mike Walker, DARPA program manager. “With the Cyber Grand Challenge, we intend a similar revolution for information security. Today, our time to patch a newly discovered security flaw is measured in days. Through automatic recognition and remediation of software flaws, the term for a new cyber attack may change from zero-day to zero-second.”

Highly trained experts capable of reasoning about software vulnerabilities, threats and malware power modern network defense. These experts compete regularly on a global “Capture the Flag” tournament circuit, improving their skills and measuring excellence through head-to-head competition. Drawing on the best traditions of expert computer security competitions, DARPA aims to challenge unmanned systems to compete against each other in a real-time tournament for the first time.

“The growth trends we’ve seen in cyber attacks and malware point to a future where automation must be developed to assist IT security analysts,” said Dan Kaufman, director of DARPA’s Information Innovation Office, which oversees the Challenge.

The competition is expected to draw teams of top experts from across a wide range of computer security disciplines including reverse engineering, formal methods, program analysis and computer security competition. To encourage widespread participation and teaming, DARPA plans to host teaming forums on the CGC website atwww.darpa.mil/cybergrandchallenge.

For the first time, a cyber competition would take place on a network framework purpose-built to interface with automatic systems. Competitors would navigate a series of challenges, starting with a qualifying event in which a collection of software must be automatically analyzed. Competitors would qualify by automatically identifying, analyzing and repairing software flaws.

DARPA intends to invite a select group of top competitors s from the qualifying event to the Cyber Grand Challenge final event, slated for early to mid-2016. In that competition, each team’s system would automatically identify software flaws, scanning the network to identify affected hosts. Teams would score based on how capably their systems could protect hosts, scan the network for vulnerabilities and maintain the correct function of software. The winning team from the CGC finals would receive a cash prize of $2 million, with second place earning $1 million and third place taking home $750,000.

A Broad Agency Announcement (BAA) with specific information for potential competitors is available at http://go.usa.gov/WqcH. Competitors can choose one of two routes: an unfunded track in which anyone capable of fielding a capable system can participate, and a funded track in which DARPA awards contracts to organizations presenting the most compelling proposals.  

DARPA also plans in the near future to issue a second BAA for proposals to develop technologies to support the competition. Support technologies will include accessible visualization of a real-time cyber competition event, as well as custom problem sets. That BAA will be available on the Federal Business Opportunities website.

The program anticipates hosting two Challengers’ Days—one at DARPA’s offices in Arlington, Va., and the other on the West Coast—where interested competitors can learn more about the event. More information, including up-to-date rules and prize amounts, is available at www.darpa.mil/cybergrandchallenge.

Partnerships Can Add Value to Prize Competitions. #PublicPrize @mkcontra @cristindorgelo


The right partner can be the key to a successful challenge competition.

If you’re planning a challenge for your agency, you’ve probably had to ask: ”Do we have the tools and capabilities to pull off this challenge on our own?”

Why we form partnerships

Often times, the answer is, “no.” But that shouldn’t stop you from pursuing solutions to your problems. Challenge managers weighed-in on partnerships at a recent community meeting, and here are a few of their tips on how agencies partner for success.

Other organizations can do things your agency can’t and complement your agency resources and skill sets.  Maurice Kent, the Lead Challenge and Prize Analyst at USAID, detailed a host of things partners can bring to the table for your challenge:

  • Resources (especially monetary)
  • Speed and flexibility
  • Functional and ready-to-use platforms
  • Capacity (reviewing thousands of submissions, for example)
  • Expertise

In describing the partnerships negotiated for the Rebuild by Design challenge created after hurricane Sandy, HUD’s Kevin Bush highlighted the need to reach out to philanthropic organizations, a university, and even the government of the Kingdom of the Netherlands in order to find the resources to execute the competition.

“Partnerships allow you to tap into expertise that you don’t necessarily have in-house”, Bush said.

Determining potential partners

Choosing partners depends on what resources your agency has at its disposal and what kind of challenge you are looking to run. Think about which companies, non-profits, and other organizations have a similar mission as your agency or to the specific goals of your challenge competition.

Consider others that would complement the initiative or have resources that government does not have. It can be as simple as getting an introduction through LinkedIn or from someone in your agency. Use your networks to see where there are connections to the companies and people you want to work with. If all else fails, there’s even the old-fashioned cold call.

If your agency wants the help of an online challenge platform to administer the competition, it has the option to use a vendor from the GSA Schedule 541 4g which has companies with expertise in crowdsourcing competitions. Vendors specialize in managing challenges in specific areas,  such as coding, apps, or social innovation, so you can find the best fit for your program.

Larry Cooper, who has helped run many challenges at NASA, said that his agency almost always immediately looks for partners that have the capability to significantly offset the workload.

“[NASA] offers the prize, but the partners do all the heavy lifting”, he said.

Cooper described a particular challenge where NASA partnered with a university. The agency needed help running a robotics challenge and the university wanted to raise the profile of its robotics program, so the relationship was symbiotic.

It doesn’t always come so easily, though, and Cooper added that NASA is careful not to force ill-fitting partnerships. ”We’ve had situations where we’ve gone out and found no one capable,” he said. “It takes a lot of effort to find the right partners.”

Legal authorization

You have the authority to use partners if your competition is using COMPETES.  The America COMPETES Reauthorization Act of 2010 grants authority for a wide-range of actions related to challenges and prizes, but other authorities can sometimes be used, too. There are also some agency-specific authorities. Work with your general counsel early and often to find the right avenues to create partnerships.

Cristin Dorgelo, Assistant Director for Grand Challenges at the White House Office for Science and Technology Policy, is a great resource. Before you give up on a potential partnership due to legal or other hurdles, contact her for guidance or to assist in conversations with your general counsel.  She can be reached at challenges@ostp.gov.

Kent’s short and central takeaway: ”The lesson to take to heart here is [to ask]… ‘What does your team lack, and what do you need to take the next step forward?’”

When you take the next step, let us know. We’d love to hear your about your experience working with partners both in and outside of government.

Prizes and Challenges Matter for Development


Prizes allow institutions to develop revolutionary new solutions, are less risky than traditional grants, and can create communities of practice

By Alex Dehgan & Aleem Walji | Oct. 14, 2013

Kevin Starr’s provocatively titled post, “Dump the Prizes,” raised eyebrows at both the World Bank and USAID’s Office of Science and Technology. Our two organizations are working to increase and improve the use of challenge-driven innovation (including USAID’s Grand Challenges for Development and its Tech Challenge for Atrocity Prevention, and the WB’s Development Marketplace) to achieve our development objectives, so naturally it’s worth considering whether these tools are worth our time.

The article is deliberately provocative and perhaps partially tongue-in-cheek. Although it puts forth a set of sound criticisms of poorly designed and targeted contests and prizes, we found that it was a very positive argument for good prize design. Prizes are not one-size-fits-all and are not appropriate for every type of problem.

Starr raises multiple issues regarding challenges and prizes—among them, costliness of competitions, ineffective selection mechanisms, failure to address the right issues, and excessive noise that obscures signals for effective allocation of philanthropic capital. These are by no means empty criticisms. Prize competitions are very hard to get right and small variations in design can have a significant impact.

Yet, hardly any of Mr. Starr’s objections are related to inherent flaws related to prizes as mechanisms to surface and spur innovation. Objectives can be clearly defined, design mechanisms carefully considered, juries appropriately selected, and feedback and prize money distributed in sufficient quantities. However, these very criticisms are also applicable to other mechanisms, including grants. Grant competitions can also suffer from poor design, unqualified juries, insufficient feedback, and significant competitive barriers. Prize and challenge competitions can be designed from the onset to prioritize solutions that incorporate implementation, scale, and sustainability.

Historical examples confirm that well-designed prizes can be effective in incentivizing the development of new products, paying for results (rather than ideas), and driving commercial actors into new markets. Although it certainly had its design flaws, USAID and the Gates Foundation’s Haiti Mobile Money Initiative successfully drove market entry and quick scaling of mobile financial services in a poor and previously neglected market, jumpstarting a generally overlooked aspect of humanitarian response—the financial systems. The Ansari X-prize helped usher in a new wave of commercial space exploration, and helped fundamentally shift NASA’s approach to travel to low earth orbit.

In the Knight Foundation Report quoted by Starr, 40 percent of non-winning participants said the process of applying was helpful in clarifying their ideas and connecting them with new partners. Half were still developing their solutions even though they did not receive funding from the prize competition. Participation mobilizes people to think about difficult questions, and allows entrepreneurs to refine their ideas and move forward.

Prizes have proven particularly effective at producing solutions to a clearly defined problem, raising public awareness, and crowding-in private capital. According to Everett (2011), the Shell Springboard Prize, for example, achieves a return on investment of between 200 and 900 percent, with return measured as the total spending from competitors and investment representing the total cost of running the competition. Prizes also are usually an effective use of resources due to low monitoring costs of fund disbursement, a relatively simple application process (compared to grant funding), quicker feedback cycles, and external peer review.

USAID and the World Bank have found that prizes and challenges can help accelerate our ability to solve critical development challenges in different ways. For one, they eliminate hidden biases of traditional grant programs by focusing on the problems rather than the solutions; this allows participants to advertise to an unlimited global solver audience, including those closest to the problems within the developing world. They also can help translate and source new ideas from adjacent technical spaces, and allow the funder to uncover the landscape of potential solutions, understand emerging trends, and create communities of interest. In addition, they have attracted new solvers and solutions to particular problems, opening the door for new entrants. Increasingly for USAID, we have seen a high percentage of awards coming from sectors that have never applied before—what’s more, nearly half of the applications and a growing share of the winners are coming out of the developing world. This is a great way to shift a development model from providing assistance to providing opportunity.

At their best, prizes are pay-for-performance mechanisms that can bring greater value back to donors and their constituents than traditional grants. But no matter how well designed and efficient prizes are, they are not a one-size-fits-all solution that we can throw at any social innovation challenge. There are risks to incentivizing organizations that seek to win prizes rather than solve overall problems. We need to use them selectively, have a clear and measurable outcome defined in advance, and use them as part of a portfolio approach to addressing problems.

We also agree that there is a scarcity of data linking prizes to actual results. One of the most extensive reports to date on prizes was produced by McKinsey & Company in 2009. Much of the report relies on interviews and anecdotes rather than quantitative measures of impact. This gives some indication of how much work remains to be done on the efficacy of prizes, especially for social impact and in the area of global development.

Ultimately, the question is what is the best decision for those who are most vulnerable—the people and communities that foundations, social entrepreneurs, and donor agencies aim to serve. The failure to employ a mixed portfolio of programs that can help the field advance in leaps rather than increments hurts the developing world—it passes on the risk of failure from those who can bear it to those who cannot. This is the standard by which we must judge our development interventions.


Read the original post, “Dump the Prizes,” as well as other responses to the same post: “Dump the Stupid Prizes, Multiply the Rest” and “Why Open Contests Work.”