Saturday, August 20, 2016

How Is This A Thing?

When talking about what sort of companies get funding from VC, I have a saying: 

Even when you take into account that VCs will fund companies more pointless than you can imagine, VCs will still fund companies more pointless than you imagined.

In that vein, I was amused today to read about Juicero, a company that makes juicers (the things that squeeze fruit and veg and make glasses of juice), and was funded to what was believed to be a total of $120 million. Yes, $120 million. I know it's been covered earlier this year but somehow I missed it, perhaps I assumed it was an April Fool's joke and ignored it, but it's for real.

So what is it? It's a $700 juicer that you buy (yes, seven hundred dollars), and then in the same way you buy different coffee pods for a Keurig, you buy different types of juice packets which range up to $10 each (yes, ten dollars). Hey, pre-cleaning and chopping organic (of course) fruit then putting it in a non-degradable packet is hard work! Pop the juice packet into the juicer, press a button, and a minute later you have a glass of juice. Then you throw away the packet, nothing to clean. But wait, there's more. The packet has a QR code on it (those square, 2D barcodes) and the system reads the QR code to compare with an internet database (it's WiFi connected of course) and see if the packet is in date - if you're in luck the system will press the juice for you just right. If not, or your internet happens to be down, no such luck and the $10 you spent will get you nothing.

So the skeptic in me sees:
  • A solution to a non-existent problem. This solves nothing. No pain point other than a bit of washing up
  • Vastly more expensive and environmentally damaging than the existing method
  • Multiple points of failure and unnecessary complexity
  • At best serves a tiny demographic
Buying the most expensive organic pressed juice in Whole Foods (you know, stuff someone has pre-cleaned and chopped and put in a plastic packet) and putting it in your fridge would be cheaper than this, wouldn't need an initial $700 investment, and you could still drink it when you the WiFi goes down. It's the sort of thing that you'd ridicule an undergraduate student for in their final year "Entrepreneurial Studies" final project, or congratulate them for the best parody startup you'd seen. But it got funded. For $120 million. How?

Industrial Design
The system looks beautiful. Just look at that sleek Jony Ive style design, it's like an iPhone on your countertop, how could you not want that? That alone makes it worth $120 million. OK, you think I'm, joking here? One of the things I've observed in the last few years of watching startup funding is this: Never underestimate the value of the mock-up, it's about the most important thing to show when fundraising. Not the prototype, the mock-up (but be sure to call it a prototype).

As an engineer, I've been more the "show something working even if it's a bag of circuits and wires, cleaning it up later is the easier part". More fool me. What I've learned is that such demonstrations press the 'off' button with investors - instead, show a mock-up or better yet the Industrial Design (ID). Best if you can put it into their hands, but an "artist's renderering" works amazingly well too. Seriously, investors seem to lose any ability to ask questions about the actual product when they're handed a piece of cardboard covered in plastic with a logo on it. "Hey, look how cool this thing is! It's amazing! All the hard work is done, all someone has to do is all the engineering/user design, validation, and testing to make it happen!"

I can see this having been part of the pitch deck to investors, a cool image and saying some ID firm run by ex-Apple designers is on it, and they'll think it's 90% done. Just be sure to accidentally say "prototype".

Market Penetration
Next I can see the slide showing some data on growth in juice bars, which if you live in places like SF or on Main Street in Santa Monica, there seem to be one every block running a 'special' of "4 for $30". I used to sit in the bar or ramen place opposite and count how many went inside during the day. If it got over a couple in an hour it was unusual, I've no idea how they survived. However, if you've got too much money then you all friends know people who 'juice', it's a health thing, and $10 for a juice isn't ridiculous so that's all OK.

Then we get the market equivalent - the Keurig. They'll have some curves showing Keurig's rise to around $4 billion in sales in 2014, and a tag line such as "The Keurig for Juice!". What's not to love about it? They sell the hardware, but then get the lock-in on the juice packets and receive ongoing revenue from that. Even better, the QR code means it won't work with third party packets, and unlike Keurig's failed attempts to create such a lockin, the fact the system is internet connected is the way they'll ensure that it can't be bypassed. Thought it was stupid that there was such a point of failure? No, to investors that's a positive! 

It's not even a system where they buy on demand like Keurig, nope here you go on a subscription and you get your supply sent every week. Better not miss your juice intake for the day, it's your health after all. They'll have that hockey stick curve of 1000 sales in year one, 10,000 in year two, and then a million in years three and out, with the consumables revenue from all those sales building nicely to make this a billion dollar company in year 5. Who wouldn't invest in that?

Supply Chain
Next they'll show how they'll corner the market in the consumable preparation, buying in vast quantities from the organic farmers and driving down the price, getting further margins there. They'll probably be something in there that shows how they can shift the content mix in each packet to use the cheapest ingredients available at the time. Something of a logistical nightmare, but it sounds great.

An Impulse Buy
It seems they aim for the "give away the razor, charge for the razor blades" approach of Gillette - except in this case they "give away" the first part for ~$700. Not a necessity and hardly in the impulse buy category, I have a sneaking suspicion they initially targeted a lower price point than this telling the investors it would sell for the price of a Keurig, with an entry level product in the $100 to $200 range, then just utterly failed to hit it. Right now I can imagine they're telling investors something like "This is the premium version, we've got a plan for cost down for the regular version!" when this probably was the standard version. It won't be the first time a CEO has demanded a product with every feature, in a really short timeframe, and then been shocked at the cost. When you're looking at time, cost, and quality, you only get to pick two, and time is never on your side as a startup.

That $700 may even be a subsidised number (though they claim not), but regardless if you assume that their COGs is around 30% of the price then you're looking at parts and labor of more than what an equivalent product retails for. They had two years to get this going, and while supply chain can take that long, without anything exotic in the design, time was not a restriction in getting this made, they should have had time to Design for Cost. Instead, I suspect a series of changing demands, feature creep, and failure to plan before initiating hardware builds made it take longer than should have been.

The Founder
And here we come to the main event, the CEO, Doug Evans. If ever you have a True Believer who is absolutely invested in this product and actually believes in what he's selling no matter how crazy, this is it. He compares himself and his product to Tesla, and that his method of squeezing gets more Chi, life force, and vibrational energy out of the juice. (Wait, vibrational energy? Maybe it can charge your phone at the same time!). 

How can you not believe in someone who can lead and inspire like this?

“Organic cold-pressed juice is rainwater filtered through the soil and the roots and the stems and the plants,” he said. “You extract the water molecules, the chlorophyll, the anthocyanin and the flavonoids and the micronutrients. You’re getting this living nutrition. It’s like drinking the nectar of the earth.”

He had the "Tenacity, Resilience, Perspiration" needed to never give up, so loved by investors. He even had Domain Experience, having run juice bars before. What does it matter this was a electro-mechanical system, consumer product, software app, supply chain, and retail play, none of which he had experience of. At least he later realised what a huge job it was to get it to market:

“I was just na├»ve,” Mr. Evans said. “I was like Forrest Gump. I had no idea what it took to make a piece of hardware that could ship to consumers safely.”

It's a pity more technically inexperienced founders don't listen to advice from those who tell them their timelines are ridiculous and that there are massive technical and safety concerns that shouldn't be ignored. 

What The VC Sees
Now imagine you're a Venture Capitalist, looking for somewhere to put your money for a possible 10x to 100x return and make your fund profitable, making up for all the other plays that tanked. Do you laugh this "Juicero" out the room, or do you make a mental list of all the positives and evaluate the risk versus the potential returns? If you did that, here's what you might get:
  • A dedicated True Believer Founder
  • A simple tagline, easy to understand 
  • A comparison business model that shows billions in revenue
  • Profit on the product, ongoing revenue from consumables
  • A 'hockey stick' revenue curve
  • No new technology needed, it's really an execution play
  • Digital lockout of third party suppliers
  • First mover advantage
Honestly, I look at that list and think "I can see why someone invested" especially if you can get them salivating over owning a part of "Keurig for Juice" early on and that it's going to be $200 a pop. If you even think there's a 10% chance it could match Keurig's $4 billion a year in revenue, a few million invested actually isn't totally ridiculous. 

So when you pitch your awesome idea to a VC and they don't invest, and then that VC pours a ton of money into a juicer company for a product that no-one is going to buy, have a look at how your business model compares to theirs. 

On paper, Juicero ticks all the boxes and makes sense as a VC investment. In reality, it's totally dumb. Do you see now why this is a thing, and you're not getting funded?

Mock-Ups, Industrial Design, and Prototypes

When you're a company showing a non-existent or early stage product off, you want to give engineers, investors, and the press an idea of where you are going with it, and so you need to have Mock-Ups, Industrial Design, and Prototypes. Each has a slightly different meaning, and while there's no firm line dividing them, I'll give the definitions that have been standard in most engineering fields in which I have worked.

Why does this matter? Because startup founders, CEOs, and PR teams might 'accidentally' get them confused and have you believing that thing they're holding in their hands really works when it's just a painted egg-carton, unless you know what to ask.

This is a very, very early stage representation of how a final product may look. It's done with no or limited user surveys and testing, engineering specs are incomplete, and is usually a basic structure made from cheap parts such as cardboard or plastic to give something to hold in your hands and imagine as to the size, weight etc of the final product. Think elementary school science fair. It's made cheaply, quickly, will change significantly before the final product, and most importantly is completely non-functional.

Industrial Design
This is something more substantial than the Mock-Up. Later in the product development stage, when the engineering specs are more solid, and there's been real user testing data to see what the users want in the product, and your marketing team has defined your target markets and how you want to appeal to the consumer, you do your Industrial Design (ID). This is where you make your product look as appealing as possible while keeping the technical needs in mind, and meeting your unit cost targets.  If you have multiple products you make sure it gives you a 'family' look, so your products are distinct - think Apple, or Dyson. There's usually a substantial amount of work needed to generate the information the Industrial Designer needs (both technical and user), then there's a few iterations of artist's renderings and 3D printed models before the final ID is made. You might spend tens of thousands to millions of dollars per product to do this well, and should be hiring specialists to do it. This is the world where the likes of Jony Ive work.

ID is a fundamental part of any product you actually want to sell, but once again it's important to remember that ID is completely non-functional no matter how pretty it looks. 

A prototype is a non-final engineering system showing some or all of the technical features of the final product, to some degree. Most products go through multiple prototypes during development, and are a critical part of the learning process for engineering, as well as developing the information needed. They are made with engineers in mind, to see performance, manufacturing and test techniques, as a platform to make quick changes and as such is often ugly, loud, expensive, and temperamental. Over time the prototypes get more refined, features more complete (or dropped), until you reach the point where you're just tweaking and then you freeze the design, and move to production. Some engineers also like to use terms such as "Demonstration System" for very early prototypes whose performance does not reach a level needed for a product but still use the same technical methods that will be improved to the point of being useful, that's a personal preference.

The most important thing with a Prototype is that to some degree it is actually functional

Suggestions to Tech Journalists and Investors
If someone shows you what they call a "prototype", try to turn it on, or ask what functionality is in the item they are holding up as a "prototype" (be really specific here, point to it, and ask what that item there can do, not what might be in the lab somewhere, as PR teams sometimes 'forget' to make that difference clear). If it doesn't work and it looks ugly, it's a Mock-Up, and if it doesn't work and looks gorgeous, it's Industrial Design. If they mumble or don't let you touch it, then it's a Mock-Up/ID and they're trying it on. Until proven otherwise, call it a Mock-Up or "non-functional Industrial Design", if it's truly a prototype then any real company will be very quick to prove that to you.

As this article on Theranos so clearly shows, startup founders have a skill of telling no lies, but making you believe there is more to their product without ever saying it, and the word "Prototype" is one of the methods of doing so. Beware.

In her 90 minutes onstage, Holmes did not tell any obvious lies. Her genius was in the strategic leaving out of information -- creating holes that people tend to fill with faulty assumptions. Instead of lying, she prompted people to lie to themselves. Understanding how to avoid being fooled by this technique is important, given how frequently it pops up in fields far beyond science. Fact-checkers often don't spot this brand of deception.

Who Doesn't Know That?

Someone sent me this image today, thinking it was really funny. It's kinda bizarre to me - Who doesn't know that Will Smith is from West Philly? It would be pretty embarrassing to meet him and say something like "I love all your shows. So tell me where are you from?". I bet he's got a really awesome expression for situations like that. Maybe something like one of these?

Yeah, I think those would be pretty close. Anyway, it reminded me that I saw Suicide Squad last week - it wasn't one I was rushing to see, following some negative reviews and that DC has really not been doing too good a job with its films lately. Batman vs Superman was not very good, very poor given the material they had to work with - you have Batman, Superman, and Wonder Woman, what else do you need? Marvel can get a great film with Ant-Man which I thought was insanity when I heard the announcement, but write a good script with good actors and you can get a fun film.

Well, Suicide Squad surprised me in a similar way to Ant-Man. I had low expectations but enjoyed it, and really wonder why it got the bad press it did. It's a slightly better than average summer action film plot, with solid performances from all the cast but definitely carried by Smith and Robbie in their respective roles as Deadshot and Harley Quinn. They really made you care about them despite their less than pleasant characters, and had a chemistry that worked. Amanda Waller is cold and calculating, in many ways is the villain of the film, as she should be. It had the right balance of action and comic relief, wasn't overly long, and left it open for a sequel, which I'd be want to see.

The Joker seemed almost an add-on, even though he is central to Harley Quinn's character. Other than to have the Joker in it, I'm not even sure why he was there. Leto does a decent performance with him, but I'm not sure that he had any material to really make him exude crazy - Heath Ledger still takes home the prize for most insane and scary Joker. 

Compared to the other summer films, better than both Jason Bourne (retread of the first three, nothing new) or Star Trek:Beyond (exceeds the low bar set by the last two ST films, but not great). Way better than Batman vs Superman, better than X-Men:Apocalypse, though not Deadpool level brilliant. Overall if you like this type of action/comedy film go see Suicide Squad and ignore the reviews. Apart from this one.

Thursday, August 18, 2016

Bait and Switch

Raising money from any source can be difficult - you have to persuade whoever has money that you're the best place for it to go, and you're up against a lot of competition. Sometimes you can do that by showing preliminary work and persuading your peers you have a sensible rational approach to moving forward, and the end result is worthwhile - most grants from the government are done this way. Sometimes you can show existing revenue and that market growth will easily allow the money you need to expand to be paid back, and a bank or other lender will see the value. When you are a public company, you have to show returns and potential for growth that make your stock look appealing to retirement funds and the public. With Venture Capital, the goal is to provide outsized returns, billion dollar companies that give 10x or 100x gains or more, ideally to make up for all the other bad bets made and have their VC fund be profitable. It's this last one, and the behaviours it encourages, that we are going to delve into a little deeper.

What Drives Cheque Sizes and Valuations?
When raising from VC, there are multiple things a company needs to take into account. First, there's how much you need to raise, (Many Series A round are in the $3 to $10 million range) and how much of your company you are prepared to part with to get that.  The combination of those two tells you where your company needs to be in valuation to make that possible, for example if you want to give up no more than 20% of your company and to raise $10 million, you need a $40 million pre-funded valuation (that's the value before you take the money) - as the post-value will be $50 million ($40m pre + $10m investment) and then that 20% is the $10 million investment compared to the $50 million post.

Most VCs have an expectation of owning a reasonable piece of the company, usually in the range of 20 to 25% with 15% at the low end, and 30% at the high end. The range of cheque sizes they are prepared to write depends on the size of their fund - how many companies they want to monitor sets the lower bound, and spreading risk to be sure not all their eggs are in one basket sets the upper end. For example, a $200 million fund may decide that a minimum of $3 million and a maximum of $6 million for Series A companies is their comfort zone, leaving some cash over for seed investment and reserved for later stage funding. These numbers vary with each VC, and with time as their fund matures - they typically last 10 years and what they do in year 1 is very different than compared to year 7.

While normally you'd want to boost the valuation of your company to minimize the dilution of your company, setting your expected valuation as very high will immediately remove a number of VCs from your possible pool of funders. For example, if you want a valuation of $100 million with a $10 million raise, then you need to find a VC not only able to write that cheque, but willing to accept under 10% of the company in return (actually, more than one since often there is a lead and then additional companies that split the deal). With a raise of $20 million the pool of VCs willing to fund is even less, but the % of the company on offer is much more palatable. If instead the valuation moves to $40 million then you still have the same pool of VCs as originally, but they will be much more interested in owning 20% of your company than 10%.

Incentives for the Founder to Push Valuation
You as the founder/CEO want to get the most money for the least equity so the goal is to match financial needs, with the cheque size of a VC that is in your area and likes you, and with an idea/company that justifies the valuation to keep VC ownership in the 20% ish area. Seeing as there is an inherent need in people to both raise as much money as possible, and believe the external validation that your company is worth an enormous sum of money, there is pressure to give the 'rosiest' view of the possibilities for your company. As I wrote before, I have even been chastised by a possible investor for presenting a 'realistic' view of revenue and told instead to show the most positive view regardless of likelihood.

A CEO is under that pressure to inflate values, particularly one without a real grasp of the realities of their area such as those who have little or no actual experience in a technical field - Elizabeth Holmes and the like, for example. They can claim ridiculous things to potential investors that no-one who actually truly understands would say, and say so convincingly because they believe it themselves. If you read VCs talk of what they look for in founders, it's almost always a 'fundamentalist religion type zeal and belief in what they are doing'. Notice in that article it's the last 2 of 12 characteristics that what most people consider critical - Domain Expertise and Integrity - and I've found (to my own detriment) that most CEOs I've worked for utterly fail in both of those. 'Tenacity' is the most important to VCs apparently, after all you don't want your investment to be thinking about reality, there might be a sucker somewhere who eventually buys the very dead horse you've been flogging. (Sorry, I mean "overcome the great difficulties being a founder entails")

Basically, VCs set the terms and incentivize what would normally be considered lying or fraud - why are we surprised when that's what we get? Moreover, it weeds out those experienced in a field who simply understand enough to put realistic expectations on what's possible, or have the integrity to refuse to lie.

Exaggerating or Lying?
When does 'rosiest view' change to 'lying'? There is no sharp line before which it's exaggeration and after which it's fraud, it's a grey area. Sometimes you are smart (or lucky) and what you claim turns out to be true, sometimes it's completely wrong, sometimes it kinda does but not nearly as well as you hoped - very occasionally it's massively better than hoped and you end up a Facebook or WhatsApp. Sometimes you might believe you can reach a metric you are claiming, and only have the resources to know for sure post Series A, and when you learn what you are truly capable of you have to 'pivot' and refocus the company on a different, usually smaller, market you can actually address.

What is it, though, when the company knew for a long time that what they were claiming was never achievable? Maybe they always knew, maybe they learned later once they had the staff and funding, but they kept going because to do otherwise was to admit defeat, and give up any chance of that greater fool buying you. So what does someone like that do? Well if we look to Theranos and Energous, the answer is 'Bait and Switch'.

Theranos Poisoning the Well for other Blood Test Tech
Theranos recently Punked the AACC and managed to give a marketing pitch for a new platform rather than actually give results on their old one on which they had raised $700 million. The old system was supposed to have been able to run up to 200 tests on mere drops of blood drawn from a finger rather than a vein, which if achievable would have been a huge leap forward. They were the darling of Silicon Valley, with huge coverage in the press for the founder Elizabeth Holmes (and all on her, not the tech). It turns out that they were not being very truthful in their claims, and now both the SEC and FDA are pursuing criminal complaints against the company as well as eight class action lawsuits from patients who received false diagnoses from the company. These exaggerated claims allowed them to raise that $700 million while still allowing the founder to maintain a majority holding of stock, for a while making her a billionaire until the truth came out.

So what did this new system do? Capillary blood from the finger? No. 200+ tests? No. Cheaper than existing? No. Faster than existing? No. More utility from a single box? Maybe. Essentially everything that made the company viable and worth investing in was a lie, and now they are trying to pretend the company is viable with a far less interesting concept, and one that was stated by experts to not having anything that didn't exist elsewhere. Had they done this two or three years ago, before actually providing patients with false diagnoses, then it would have been a 'pivot' - a company that made a noble and commendable effort but didn't quite work out. But they didn't, they kept the illusion of capability going far beyond when any sane person would have dropped it, and fully moved to the realm of "Bait and Switch". Turns out they get to keep that $700 million despite at some point having moved from 'exaggerate' to 'lie' in their claims - way to reward bad behaviour.

That VCs burn their money (some of which comes from pension funds remember) on a stupid bet is one thing, partly that's what they do, but because they both allowed and incentivized Theranos' behaviour, that target of a fast, cheap, small, versatile, consumer friendly blood testing was the norm for anyone else raising money in that area for the last few years. Imagine you had a product that did literally half of what Theranos claimed, and you pitched to VCs who kept rejecting you because they expected and demanded a company that exceeded Theranos. An honest founder couldn't pitch that, a dishonest or naive one could. By Theranos continuing the charade of their viability they made it harder for those legitimate startups to raise anything at all due to unrealistic expectations. It's great for a company to kill their competitors but not what we as consumers or investors (or I assume LPs in a VC) want.

Energous and the Pointless Product
Energous are wireless power company who claim to use RF (like wifi) to power small devices like phones. No independent third party has ever validated their system or performance, and some claim they are simply using a "Time to Carrot" approach to constantly keep investors thinking that the pot of gold is about 12 to 18 months out, and just put more money in. They have claimed up to 4 Watts at up to 15 feet from the transmitter but there are so many skeptics, myself included, that look at the physics and maths and show that what they claim is simply not possible. After going through an IPO, raising millions of dollars and the top three executives paying themselves almost $5 million a year total with no product or revenue, how do they answer their critics? By releasing a product, but so simple and with specs so low that it is pointless, and calling it "mini" - and now they claim they have a product, and it's just "the big pot of gold is just around the corner...".

The mini-WattUp is a small USB sized device claiming to charge devices, that needs to be in contact to an inch away (not 15 feet), and charges at a rate that would take days at best to fully charge a phone, but it does have FCC approval (because it does nearly nothing). It's like making a car for a soap-box derby and claiming that next year you'll be competing with Tesla. It achieves the goal of continuing the illusion that there is real technology, real hope of a full scale version which will always be "18 months away".

Had Energous tried to IPO the company based on the mini-WattUp then they would have fallen flat on their face - nothing interesting, useful, or better than the competition (by far). If the goal though is to raise the cash, milk it for as long as they can, then a "Bait and Switch" keeps the money flowing, and that's the most important thing. <sarcasm>It's a pity as it destroys any market for real at-distance wireless power companies.</sarcasm>

Speaking of Other "At Distance" Wireless Power Companies
For some reason, I wanted to remind everyone of uBeam's claims of how they will be wirelessly charging at a distance, and by the end of 2016 (only a few months left to wait!). Released specs are:

I can't find anything on safety or efficiency that's public from uBeam, though there are some well written articles on the safety aspect. Just keeping this in mind for comparing to the product uBeam must be releasing soon.

Bait and Switch
So looking at Theranos and Energous, if you're wondering why they make the claims they do which have never been backed up by evidence, it's because they've been paid millions of dollars to do so. The system simply encourages it, and it's basic human trait that when you reward a behaviour you get more of it. VCs by their funding approach are selecting for founders and CEOs most willing to exaggerate, and in some cases willing to lie. If we want to see less distortions in our allocation of capital and see it more go to genuine, viable technologies, then something has to change. In large part, one of the culprits is the tech media, who simply reprint PR scripts they are handed, and give up actual ability to criticise in return for access. We need more willing to ask the hard questions in the way John Carreyrou did of Theranos or Lee Gomes did of uBeam, rather than just parrot a PR line handed to them without question.

Until that happens, expect more companies to raise large amounts on the unfeasible, and then finish the "Bait and Switch".

And finally
Just as I was to publish this, I read a fantastic piece in the The Atlantic by Adrienne LaFrance about "Access, Accountability Reporting and Silicon Valley" which says a lot of what I've been trying to say on the media coverage of tech firms, but far more eloquently. I highly recommend it.

Saturday, August 13, 2016

Pick Two, But Only Two

There's a saying in engineering:

"In any product development project you can have it good, fast, or cheap. Pick which two."

You tell this to most people, even non-engineers, and they get it. Something has to give, and choices get made in development. So why is it so hard to put into practice?

Good and Fast:
If you want a quality product developed quickly, then pay overtime, consultants to help, fab houses to expedite, and develop without consideration of materials or processes used no matter how expensive. There is, however, a limit to how fast some projects can be done, and infinite money won't get a project completed instantly - some things just take time. The 'C' level often fail to grasp this - they say they know it, but actions indicate that very often they don't.

It's common to fall for the "Mythical Man Month", where the expectation is that if you double the staff on a project, development time will halve. At some point, this is like saying "Nine women can make a baby in a month", and in fact the management overhead of too many people on a project can slow things down. This is the equivalent of computing's Amdahl's Law, where some tasks can be parallelized, and some are inherently serial - in the end, it's those serial tasks that limit how fast things can be done even with infinite computing resources.

Under pressure of timeline, management often cut what they are told is critical - design verification testing, engineering verification testing, initial user testing - I could say because they have no understanding of the need, or they don't listen when told how important they are, but often it's because they can hope that things will work out, and the blame will be on others if it doesn't. "You should just do it right first time" I have literally been told when objecting to that route. That some problems just have a minimum time and effort to solve, is a hard reality to accept for some. Blame, ultimately, ends at the C-suite, but until that day of reckoning, the minions are suitable cannon fodder for blame. 

Heavy pressure on management also makes them susceptible to the sell from the outside group who swears they can do it faster. better, cheaper than the internal team - "Just sign this contract and we'll take care of it all!". It's nice to be able to sign with someone who promises a solution, when your own team keep telling you of major issues and delays, but when it does go wrong it's the internal team that's going to clean up the mess, and for a lower hourly rate than the outside group was paid. Importantly, it doesn't go unnoticed by the engineering side that they were not trusted, and that does not help future interactions.

One particularly frustrating situation for engineers is when they deliver what's asked for, a quality product in a timely manner, only to be berated for it being too expensive when only weeks before "cost is no object!". Stating that you were (just) within the cost target set falls on deaf ears, and you're sent back to lower the price - without, of course, sacrificing quality or with a delay (which there will inevitably be). I've walked away from those meetings hearing the remaining executives saying "Engineers just never understand cost, it's beyond them." when in actuality we understand it perfectly. Cost is a constraint, like size, or weight, or performance, and we'll work to hit that. Cost targets can move during development, but poor choice of that target is a failure of the business side, not engineering.

Fast and Cheap:
Now, if you want something fast and cheap, then as long as the product doesn't have to be very good (say, falls apart on second use, or doesn't actually do what is claimed) it's achievable. Engineers hate this one, by nature they want to make a good product, and they know they'll be fixing things later on anyway when it will be much more expensive to do than if it was done correctly the first time. Software is often released this way, with the hope that the application is 'just good enough' or that the customer can be placated while fixes are put in place. 

With a non-critical product, and an understanding customer, this can be a viable route for a business, especially with software where patches and updates can fix problems and add needed features quickly. On the hardware side, this is more applicable to disposable or very low cost products, physical objects don't tend to improve with a patch. Despite this, such a mentality has started to make its way into medical related devices such as with Theranos, and other safety critical areas such as automotive - I've written about this in more detail here and here

With hardware this method is almost guaranteed to accrue technical debt - problems that cost a lot more to fix in future that to fix now. Hardware that fails in the field regularly and is returned, is a nightmare for warranty cost as well as reputation and returning customers. And guess who gets the blame for a product like that?

Good and Cheap:
Good and cheap (or affordable) is the rarest combination, it's hard to give the time and resources needed to do a job well first time. The pressure to release products to earn revenue, or placate investors, is intense. A few companies can do this, for example Google or Apple with special projects like autonomous vehicles, but in most cases it's very, very hard for normal companies to do this. If you can give a team time to solve problems and iterate, design for cost from the beginning, investigate alternate routes, multiple rounds of customer testing, and the equipment they need, then you can deliver great economical products. Sadly, it's the exception, not the rule.

Good, Fast, and Cheap:
Think you're an exception to all this? Go ahead, good luck, you may be the exception. Odds are that you're going to end up with a product that's late, not quite as good, or more expensive, than if you'd just picked two. If you're ignoring your engineering team who've already told you this, then maybe you're the CxO of a company I've worked for...

Which Two to Pick?
So what is the solution to this problem? There is no one answer to that, every company, every product, every team, every market is different. Getting a product to market, at quality, at cost, and on time, is a monumental task, for both the business and engineering sides of a company. The companies that I've seen succeed the best at this set a culture that is adhered to even under great temptation to move away from - and that consistency has to come from the CEO. This culture usually revolves around a few key points:
  1. The business side sets clear and realistic goals for the engineering team - performance, cost, timeline - and stick with them. Engineering need to be able to trust the business side and what they say.
  2. CEOs need to understand that their word does not overcome reality, and that timelines in particular can be impossible to shift. Business needs to trust that when engineering say a timeline or cost target isn't achievable, they need to re-evaluate that metric.
  3. Engineering need to communicate realities and issues to business in a clear manner, with an understanding that sometimes terms like 'at risk' don't always mean the same to each side.
  4. When things go wrong - either engineering performance or business predictions - there needs to be honest and open communication with a mind to fixing the problem, and not a witchhunt looking for the target to blame.
It seems simple, when it comes down to so few things - each person doing their job, open and honest communication, and teamwork to fix a problem when things (inevitably) go wrong. All it requires is competent leadership with a basic understanding of reality. Why is it so rarely seen?

Friday, August 5, 2016

Follow the Money

It seems I'm not alone in my disgust at the leadership of the American Association for Clinical Chemistry (AACC) and how they allowed Theranos to abuse their conference to try to pretend they have any legitimacy. The FT reports that members of the society are apparently resigning in protest at not just giving Theranos the platform they did, but the way the normal rules were bent to eliminate the chance of actual tough questions.

Dr Andy Hoofnagle, a member of the organising committee, said he and several of his colleagues had “fought really hard to prevent” Ms Holmes from appearing but were overruled by the AACC president, Patricia Jones.

“I’m removing myself from the committee and don’t intend to pay my dues next year,” he added, in effect announcing his resignation from the association.

Others were not happy with the decision and called the society President, Patricia Jones, "stupid" but decided not to resign as she leaves office next year. Jones defended her decision, deriding the very idea that even though she'd had to fight her AACC colleagues and force the Theranos presentation on them, anyone could possibly even consider it inappropriate.

This is the same person who introduced Elizabeth Holmes before her talk on Monday, and as she ceded the podium to the Theranos CEO could be heard to say "You're going to be awesome."

Now some have defended that as something a moderator does to help keep a presenter calm and perfectly normal. As someone who has done that role of introducing speakers, I can tell you that no, it's not normal. You'd be hard pressed to consider doing it for a first year graduate student at their first talk, let alone the CEO of a (once) multi-billion dollar company.

I found myself wondering why Patricia Jones was so willing to sacrifice the reputation of her society, or how she could be so naive as to allow a wolf in sheep's clothing to abuse them so badly. As I was talking this over with a friend of mine, a very Sarcastic Brit, he made the following comment:

Follow the money

And then I remembered that this April, following terrible media resulting from the lawsuits and criminal probes it faces, Theranos created a new Scientific and Advisory Board to aid in "advising Theranos regarding the full integration of its technology into routine clinical practice, and publication and presentation in scientific journals and at scientific meetings.". At the time I wrote a piece stating that it was potentially an attempt to buy legitimacy, but now I wonder if it was much more than that.

Of the eight members of this Board, four of them once held the same position Patricia Jones does now - President of the AACC. The board page at the Theranos website lists them as:

Susan A. Evans, PhD, FACB
Ann M. Gronowski, PhD, DABCC
Larry J. Kricka, D. Phil, FRCPath
Jack Ladenson, PhD, DABCC 

Did any of them apply their influence to get special treatment for Theranos and Elizabeth Holmes at the AACC conference? It wouldn't be surprising to make introductions, it's a common part of the role to help with connections, but getting special treatment from a scientific society and effectively endorsing the company would be going too far.

But most importantly, does Patricia Jones hope that this time next year there won't be four out of eight board members as former AACC Presidents, but five out of nine?

Tuesday, August 2, 2016

Theranos' AACC Presentation: Lack of Data Proves Doubters Were Right

Yesterday afternoon, Elizabeth Holmes, embattled CEO of Theranos, took to the stage at AACC to finally reveal the science and data behind their products and tests. She started with a heartfelt apology for the mistakes the company had made, for the misleading information and marketing both to patients and investors, and proceeded to finally show the actual data and methodology of their tests, warts and all, and then pleaded with the community to help the company move forward and realise the initial promise of Theranos, of multiple, accurate, rapid tests from a single drop of blood. The crowd rose to their feet at the end, and applauded her honesty and humility.

Or at least that's what happened in an alternate reality where shame over fraud and scientific honesty are foremost in the minds of startup CEOs. Instead, what we were treated to was a marketing presentation for a future Theranos product, no apology, no real data, and evasion of any substantive questions. If you have 90 minutes to spare, watch the whole thing, it's a masterclass in abusing the platform given by a professional organisation to do marketing and try to give the company a veneer of credibility to those not versed in the subject.

They didn't just fail to reach the low bar set for them - they didn't even attempt to reach it. To list out what was wrong with this presentation would take a book to cover fully, and there are multiple articles detailed issues with it such as the Wall Street Journal, Wired, and the New York Times. I'll try and cover the main points below - for those of you looking for a history of Theranos, there are some summaries here, here, here, and here.

As someone who has actively organised and managed scientific conferences and talks, I'm stunned by what AACC leadership allowed Theranos to get away with. Theranos committed multiple major faux-pas, and Holmes' celebrity status allowed her to get away with things no other presenter would have been. In a scientific conference the goal is not just to present data, but to withstand the scrutiny of your peers and as such the question component is just as important. The talk was set for 90 minutes, split into 45 minutes presentation and 45 minutes questions. Holmes gave a very polished and well rehearsed talk for an hour - clearly and deliberately done to cut 15 minutes of time from questions. This is an old tactic, commonly used by presenters who fear questions, and one an experienced chair/moderator knows to look for. The session chair was left with the choice of cutting them short and leaving out the component everyone thought they wanted to see, the promised (though never really delivered) final section on actual results comparison, or allowing Holmes to cut into the question time. They went for the latter option, a difficult call to make and one I can understand them doing.

Holmes also committed one of the worst crimes - not presenting what was claimed in the abstract, and instead giving a marketing talk for an upcoming company product. In the Wired article linked above, Stephen Master, professor of clinical and lab pathology at Weill Cornell Medical College. “This sounded like a talk from a manufacturer.” Conferences have commercial presentation sections for that type of talk, and anyone who gives a marketing talk in a scientific slot will never be invited back. AACC leadership should have made comment on this after the talk, stated it was unacceptable, and in being silent they are effectively endorsing this approach.

Her talk was slick and obviously well rehearsed, and covered things in a way that would be ideal for fundraising, but inappropriate for a scientific talk. She used buzzwords that CEOs seemingly have to say, without apparent understanding of their meaning. "Inflection Point" is one of them, which is code to a VC that "we're about to have massive sales, real soon, honest" but everyone else knows is just cover for poor performance to date. For example, Holmes uses it at 42m40s correctly to describe a test function, but tries to pretend at 56m20s that sales of this future miniLab will soon boom. Meredith Perry, Founder and CEO of uBeam that similarly has been questioned about the validity of its technology, said in September 2015 "We’re at a massive inflection point. We are about to head into a completely new phase of growth.” but nearly a year later no sales, no demonstrations, no products. Beware the CEO who promises an inflection point, it's nearly always a false promise.

Instead of answering the many questions scientists had, Theranos presented their future 'miniLab' device, a 'cloud connected' unit that may play well with VCs but fell flat with doctors and scientists who actually use the equipment. Geoffrey Baird, an associate professor in the department of laboratory medicine at the University of Washington, commented in John Carreyrou's Wall Street Journal article:

Every piece of technology they presented has been known for many years, and exists in other platforms largely in the same configuration, or in some cases in much more compact form in competitor’s platforms.

So even the future (non-existent) product they are trying to distract us with does not even move the current state-of-the-art forward in any way. In the initial questions following the talk, all done in comfy chairs and a collegial atmosphere I have never seen at a post-presentation question session, one of the moderators called Holmes on the pedestrian nature of the equipment and the departure from the world-changing claims Theranos had raised $700 million with. Masters noted that the original claim was "Many broad tests, 70 tests from a couple of drops of blood - this falls far short of that... I can buy a point of care instrument today that does a finger stick lipid panel." and received much applause from the audience. Holmes then did what was standard for the rest of the question session, talk a lot, but fail to actually answer the question asked.

Holmes clearly had memorized the line "What we wanted to do here today was..." followed by a few options such as "begin engaging with the community." or "present our future plans to the audience." No-one corrected her that this differed from the abstract of her conference talk, but you could tell people were unhappy with those non-answers.

This continued with all questions designed to elicit information such as "Why present data for potassium with venous not capillary blood?" (a key question about their methods) the reply was "What we wanted to show was..." followed by no answer. When asked about false positives and concordance rate, the reply was that their technique was "Very good" and then no answer, even when asked a second time. It was clear no real answers would be forthcoming. 

Whenever pressed to answer questions on previous methodology based on their Edison systems (which were never mentioned once in the talk despite being the main company product to date), Holmes gave the most frustrating answer of all:

In the appropriate forum, we’ll address those. But today we’re hoping to be able to engage on a scientific exchange on this platform.

A canned answer, giving no indication of when that data would be forthcoming or what that appropriate forum would be - ridiculous given that there probably is no better place than the AACC, and that audience, to address them. The real data everyone wanted to see, the topic everyone wanted to cover, was dismissed in those two sentences, and not a peep from AACC leadership or moderators on that. 

As Wired magazine points out, Theranos just 'pivoted' - that's when a startup company realises their past approach and technology just can't do what it was claimed, and try to shift to another product or market in an attempt to save the company. It's clear from the reactions that no-one was impressed by the miniLab equipment, and that had this been what was pitched to investors, there would have been no $700 million raise. As one of the moderators, Master, said in Wired:

Does this live up to the hype? The answer to that is no.

Finally, no-one raised the most obvious question which should have been:

"Since you (Holmes) have been banned by the CMS from running a company doing blood tests, and you have made it clear that you will not leave the CEO position, how will the Theranos be able to offer this test and equipment?"

That in itself would have been worth the non-answer answer that would have come.

So in summary, Theranos avoided answering any of the questions scientists wanted answered, have abandoned their old technology, and shifted to a new unproven technology that has no advantages over existing available equipment. They truly live in another reality if they think this will be enough to save the company.

Equally importantly though, AACC allowed its prestigious name to be used by someone being investigated by both the SEC and FDA for fraud, to not present real data but instead a marketing talk and to avoid substantive questions, and to fail to press even on the soft-ball questions it asked. Holmes may not have the capability to show remorse, but the AACC should be ashamed for what it has enabled here.