ARTIFICIAL INTELLIGENCE: Explaining Black Box Algorithms to Avoid Discrimination

Speaking of Amazon, news broke that the company had built out an AI-based recruiting tool that was supposed to help it rank candidates at scale. They certainly are not the only ones -- the tech startup space is littered with applicant management and analysis software, especially given that employees have many more jobs on average than in prior decades. What this AI did, however, was systematically discriminate against women, down-weighting resumes that included the phrase "women's" in descriptions or candidates that came from all female colleges. This result came unintentionally from the underlying data. If you correlate the language in thousands of employee resumes, you will get the status quo, which is that on average the Amazon employee, or any tech employee, is more likely to be male. Another artifact that mattered is the way candidates used language itself, which can be gendered in output.

Other examples of unethical AI are plenty. For example, image recognition algorithms make an error of 3% for white male faces but 30% on black females faces. Or, when used in automating sentencing criminals in the US, algorithms punish minorities more harshly. Or, when underwriting credit, AI disfavors historically disadvantaged protected classes using Zipcode. But the math isn't wrong -- it is in fact painfully correct. These outcomes are a mirror to how things are, not a solution for how we want things to be. Yet AI will be used regardless. Just this week, Lloyds adopted speech to text passwords for telephone banking, replacing pins with the sound of a customer's voice. Will this service work better for majorities and not minorities? Further, such security can be gamed using pre-recordings, or generated voices. Similarly, image recognition can be gamed with photos or by twins.

This is why we are excited to see two initiatives make the news. The first comes from the MIT Lincoln Lab, focused on machine vision. The software builds a visualization based on how a neural network sees an object, highlighting which parts and features of the object drive a particular decision. The picture below shows how the computer detects "large metal cylinders", first looking for size, then for materials and finally for shape -- each  highlighted by importance-ranking heatmaps. The second comes from IBM, called the Trust and Transparency service. In an example around insurance claims automation, the company shows an explanatory overlay on the AI that points out the probabilistic weightings for different drivers of an approval/rejection decision. A human analyst can then understand why the machine made its judgment. We think such tools will be required for any serious AI company.


Source: Reuters (Amazon), IBMMIT Media Lab, Business Insider (Amazon), FS Tech (Lloyds)

CRYPTO: A utopia that can buy its own Sovereignty

Power. Sovereignty. Utopia. A recent piece from Daily Fintech points us to Sol, the Puerto Rico crypto billionaires experiment; Bitcointopia, an experimental city in Nevada; and Varyon, an artificial island off the coast of French Polynesia. These attempts at a new world are ostensibly about cryptocurency adoption, but their precedents trace to the DNA of humanity itself. A utopia (or dystopia for that matter) is a dream of the world defined by its impossibility. It may be a guiding light, or it may be a warning, but it is not reality. To carve out a utopian experiment has immediate connotations – cult-like, counter culture, naïve. See the utopias of Russian architects in the clutches of the Soviet Union, building cities on paper that could never be, or today’s techno retreat of Burning Man, where billionaires recreate Mad Max landscapes to feel human outside their corporate castles.

For most of human history, the frontier was a real place. It was the place where water dripped off the world into oblivion, the place where the pantheon of Gods looked down on mankind, a land unconquered by ships and swords. As humanity lifted the fog across the globe, the physical frontier disappeared. Sure, hard military power still applies in redefining borders between neighbors. But there is no more room left for Manifest Destiny, other than in our imagination. From this mental frame, we bring forth economic and technological frontiers, conquering not the Earth, but ourselves. But let’s not be fooled. Sovereignty, that embodiment of lethal force in the hands of the law, may have maxed out across the geography. But control of sovereignty can still be bought. After all, we are human, and our power comes from belief in the source of that power. Economic and technological conquering results in the re-shaping of sovereignty. Facebook’s 2+ billion users are larger than any country on the planet. Does it’s soft power echo across governments? You bet it does. Tech giants spend millions per year in lobbying, driving their desires into the body politic.

At the heart of every tech company with aspiration to go public is a utopia waiting to be unleashed. Uninspired by the political realm, we burn our hearts into capitalism. And these are beautiful creations. But once they taste power over people, once billionaires hold monopolies (e.g., from Bezos to Bitmain), utopias start wanting an army and a police. Small sovereigns and peripheries of large ones give first, yielding their regulatory apparatus to help perpetuate the new paradigm. Want to launch a crypto investment vehicle wrapped in a legal veneer that purports to be of equal stature to European (Malta), American (Puerto Rico) or British (Gibraltar) law? Or maybe build a new bank under custom-made Lithuanian regulation? But it won’t be enough for Crypto, which is not merely information flow, but information married with money. Crypto will buy its way into being a sovereign, if it can’t persuade the incumbent ones to let it be.


Source: Daily Fintech (Crypto utopias), Utopias (SolBitcointopiaVaryon), Lithuania (Fintech bank), Palace of the Soviets

SOCIAL MEDIA: New Solutions for the Attention Economy

Source:  Newsweek / Slate

First, the attention economy is the lifeblood of large tech firms. They ingest human attention through social or entertainment ecosystems, and sell that attention, targeted through personal data, for advertising revenue. According to eMarketer, Google and Facebook generate over $100 billion in net revenue from digital ads. That's about as much as Alibaba, Baidu, Tencent, Microsoft and everyone else combined. Most tech platform building activities from these companies is a way to grab personal data and repackage it as a product, rather than charging a consumer for the product. None of the privacy initiatives to undo this, like Ello, have worked until now.

The law of conservation of energy prevents people from creating perpetual motion machines. In a similar way, attention is a limited resource. Attention has scarcity, and can be turned into a digital asset that is traded and used as currency. Projects like Steemit, Zappl, and LBRY are networks with consumers and producers. Consumers have mechanisms for rewarding content with their interaction, and producers get paid in native tokens. The source of the currency is based on proceeds from an ICO, or from speculation on the attention coin. Others, like Brave / Basic Attention Token or GazeCoin have a mechanism for capturing attention as part of their product. Whether it is powering micro-crypto transactions through a browser, or by recording the view of a user on a particular piece of content, these projects automate the curation aspect. And then there is the swath of ICOs, like Kodakcoin or Poet, that are trying to build crypto aspects into the content itself. All three approaches challenge algorithmic advertising as a the default monetization model of the web.

Quantifying this at such an early stage is tough. Facebook has over 2 billion monthly active users, while Steemit has 150,000. If we rewind back to look at young Facebook, it had 1 million users in 2004 and 6 million in 2005. So crypto social media usage is .01% relative to current Facebook and max 10% of Facebook at the same stage. From a value perspective, one crude metric is to look at the implied marketcaps -- STEEM around $1 billion, BAT at $400 million. We can think of these as aspirational market prices for the value of the attention economy that can be enabled by these systems. If global digital net advertising revenue is $200 billion or so, at a 10% discount rate, it represents an asset of $2 trillion. This implies about 0.1% of expected attention economy value, as priced by the markets, is in crypto .

But there is another solution, and it is the answer to the question posted at the start -- browser-based crypto currency mining. On visiting a website, the user's browser is hijacked (or willingly given) for the purpose of mining the privacy oriented coin Monero. Because Monero is CPU and not GPU intensive and is untraceable, it is the perfect candidate for sites like The Pirate Bay (already doing it) or the New York Times (should be doing it) to monetize their content. In a sense, this is a frictionless, effortless way to actually get readers to move away from the assumption that content is free, and also reduce the friction inherent in paywalls, adoption of blockchain-based software, or re-engineering of content packages. We never gave consent for the big tech firms to take our data -- do we need to consent to hand over our CPUs?

ARTIFICIAL INTELLIGENCE: Inequality, Unethical Robots and Unemployment

Source: KKR

Source: KKR

Something's going on when MITKKR and Bain & Company all publish on inequality, automation and ethics breaches resulting from technology. The KKR report highlights that GDP growth is likely to slow in the West resulting from an aging population and a displaced worker force due to declining manufacturing employment. Productivity may go up on average as augmented humans become more efficient the work place, but the long tail of regular people who do not have gainful (i.e., cyborg) employment will increase. Re-skilling has not kept up, and the down-case is that 30% of all activities across 800 occupations can be replaced by software. We have argued before that digitization results in 50% revenue declines in industries that are fully transformed. 

The Bain study reiterates the main points -- automation of the US service sector has the potential to be a catastrophic event in terms of human employment (20%+ down), a conclusion previously reached by McKinsey. The corollary is that this will happen much faster than the transitions out of farming and manufacturing. Therefore, volatility in capital markets will increase, driven by the the middle-class being hollowed via the power laws of software and capital rents, thereby negatively impacting many industries targeting them as consumers. And the headline takeaway is a $5.4 trillion GDP shortfall by 2020. Yikes.

And the robots we make aren't even nice to us! In a study of image recognition artificial intelligence systems, top three commercial software packages had an error rate of 0.8% when determining the gender of a light-skinned man, and a 20-24% error rate when analyzing pictures of dark-skinned women. This bias comes from the underlying data, which does not have enough diversity to correctly teach the software, and the bias in the data comes from bias in the development team and organizational culture. Now imagine that such systems are used by police to identify criminal suspects, the way China is doing today. Would discrimination go up or down, if all people of a certain type are imprecisely profiled by software? Or imagine connecting these data sets to access to financial services? 

So what solutions can we imagine to this dystopian television series? Well, one idea is Universal Basic Income, which is being explored in Finland, Scotland, and the UK. Or maybe Amazon and JP Morgan will save us.

Source: Bain

Source: Bain

Source: Bain

Source: Bain

FINTECH: The Value of Centralized Vision

Source: SpaceX

Source: SpaceX

Elon Musk launched the Falcon Heavy rocket into space with a Tesla car carrying an astronaut dummy, blasting David Bowie. His competitors spend about $500 million per launch, because the very expensive boosters which get the payload away from Earth's gravity fall back down and explode. Musk's boosters are smart -- they can land and be re-used. As a result, his cost is $90 million per launch. Good luck competing against a 5x advantage. 

We don't bring this up to point to the cult of personality, but instead to the power of a determined, clear vision backed by well-funded and functional organizations. While Fintech has been about democratizing access to financial services for customers, Crypto has been about decentralizing production to the community. Decentralization is useful and can create certain desired attributes, but it is not always strictly better. The Falcon Heavy did not come from design by committee, crypto consensus mechanisms, or votes from survey groups about what features customers would like. It came from reverse-engineering the future based on a purpose, levered with human and financial capital.

In Finance, the West is failing to have any coherent vision of the future. Few of our financial leaders have articulated anything close to an artificial intelligence or crypto strategy that coordinates across divisions to build a coherent future. Nobody has bet the farm. Compare for example with China. Hyperledger's executive director Brian Behlendorf recently discussed why operational progress in blockchain adoption among existing industry is far ahead in Asia. Money is moving through productions systems. $2 billion is being spent on artificial intelligence research and education. Americans are still debating coal.

In that light, we have to give credit to Overstock, which continues to move in the fintech direction that Amazon is avoiding for now. The online retailer has 40 million unique visitors per month. They can buy goods using Bitcoin, and now for $9.95 a month they can get a roboadvisor offering. That's right -- in addition to launching it's own blockchain-based trading system tZero and pursuing an ICO, the company is offering investment product portfolios constructed from baskets of stocks. The custody comes from Apex, and the algorithms from FusionIQ. And who knows -- maybe Bowie's playing in the background.

FINTECH: Two-sided Ageism

Source: WSJ,  CMO

Source: WSJ, CMO

Last week we talked about how Fintech (and startup culture in general) has a deep Femtech failing that needs to be addressed. So let's talk about the other diversity problem in the industry -- ageism. In a recent survey, Fintech Circle Institute found that (1) 94% of financial services professionals suspect their colleagues of using buzzwords like “Blockchain” and “Artificial Intelligence” without understanding them, and (2) 84% said colleagues with advanced digital skills and 5 years of financial experience are more deserving of promotion than those with poor digital skills but 10+ years of financial experience. Tearsheet has a great article on framing the problem: Senior executives have to retrain for a Silicon Valley-inspired financial services industry or square off against younger, cheaper Millennials that come digital first. These issues are turning into institutional hiring/firing decisions, which in turn lead to age-discrimination lawsuits against firms like Bank of the West, with ajudgment of ~$1mm awarded to a wrongly terminated 61 year old executive.

On the other hand, take Jamie Dimon's unexamined proclamations on cryptocurrency (Laura Shin of Forbes does a spectacular job of parsing where he goes wrong). The future is built on a different DNA, and yet Jamie is using his global platform to stubbornly perpetrate outdated thinking. Or look at the mind-boggling ease with which former senior bank executives are raising money for startups with minimal traction. Former Barclays CEO Antony Jenkins received £34m for his bank-as-a-service infrastructure play, with support from Oliver Wyman and Ping An. Sallie Crawcheck raised $35m for Ellevest, which only has $55m in assets under management, far less traction than what FutureAdvisor, Wealthfront or Betterment had to show for a similar size check. If the firm you led perpetuated the problems in the industry, why should venture capital go to your new company, and not to a Millennial with a moonshot idea?

There are no easy answers, but here is an attempt. Build teams that understand your customer, both where they are today and where they will be in the future. Smart thinking and a desire to innovate have nothing to do with age -- as this 94-year old inventor of the lithium-ion battery teaches us. But the half-life of industry experience and knowledge is getting shorter and shorter, so an open mindset that encourages changes in the status quo is required of us all.

FINTECH: SoFi and Uber Are a Symptom We Must Fix

A deeply disappointing and embarrassing account of SoFi corporate culture was exposed last week. Starting with CEO Mike Cagney and through multiple levels of management, there are serious allegations of sexual harassment, discrimination and disrespect towards women over whom SoFi executives held power. Cagney resigned abruptly to give the company a chance to move past the accusations. The story would be more shocking, were it not a blow-by-blow repeat of Uber. And the fall of Zenefits, driven by aggressive expansion skirting regulations and a frat sales culture, comes to mind.

In defense of its core business, SoFi has published strong statistics around performance in the last quarter -- $134 million in revenue, 60% YoY growth, $3.1 billion in loans. The dangerous implication is that high quality financial performance is what matters, and personality scandals are acceptable (or perhaps do not reflect on the corporate entity). Do we really have to accept cowboy entrepreneurs as a requirement to build high-growth innovative companies? Is a toxic startup ecosystem an acceptable byproduct of chasing growth and hitting revenue numbers?

Change starts from calling things like this out, and understanding that companies operate in a society, not in a vacuum. They are collections of people, and their mission and culture reflects the example set by leadership. Therefore -- leadership matters. Further, when attention is scarce and product choice is infinite, creating a disrespectful culture is terrible business. Consumers can and should vote with their wallets -- Commonbond will be glad to have them. And last, as Fintech companies build increasingly more human judgment into software (think blockchain and artificial intelligence), these biases will become massively amplified. Powerful tools should be built by teams with diverse backgrounds, genders, ethnicities, and wealth levels, or risk perpetuating the mistakes of the old world.