Doing analytics, data science and business intelligence in either type of company can be extremely rewarding, but there are critical differences to understand. At its most basic, think of it as the difference between racing in the Formula 1 and racing in the Mongol Rally. You can go mind-blowingly fast when you’re on a paved track, with a specially made car. But that car can’t even drive itself to the competition, much less find an exciting shortcut on the raceway.
In the end, a Grand Prix driver is also just one piece of a giant machine, that involves the sponsors, management and pit crew. A Mongol Rally vehicle sets off along whatever route the driver chooses, empowered (and required) to think up solutions novel to whatever opportunities and emergencies might arise.
In the startup context, it’s unlikely you will run into issues between different departments. Often, entire departments will consist of one or two people. However, you’re often working with much less complete data sets. Even if you’re collecting a lot of data per day, it will not have very much historical evidence to reference. Often you will be leveraging third-party data to try to extend your visibility outside your company’s limited experience. Third-party data can bring incredible insights, but it comes with its own challenges, particularly when you’re trying to integrate two different outside data sources.
Government data can be some of the best and worst from this perspective, representing an incredibly deep data set, but it’s often stored in bizarre file formats and with painfully dense reference material. For example, data.gov gives access to a dizzying array of sources (192,927 at the time of writing), but each is formatted differently and represents a considerable learning curve. That said, several UK companies have done impressive things with their government’s data.
Vance Fitzgerald (blogger and expert on the Raleigh-Durham startup scene) points out that some startups are taking another route, building their business around data supplied by their customers. This presents fascinating opportunities and challenges. You’re never going to find more relevant data than that supplied by the customers themselves, but you also have to contend with all sorts of formatting issues and exceptions, often on short notice.
It’s not like enterprise data lakes are any less difficult to wade through. For an organization that’s been around long enough, standards have evolved in different directions, like Darwin’s finches, to the point that they can no longer interbreed and produce beautiful little data babies. At least in the corporate context, you have access to the experts behind the data, and they can help you understand where it came from and (if you are persuasive enough) even help you modify the data collection front end so that you’re getting what you need to work with.
It’s often said that work in an enterprise moves less quickly than in startups. That is true if you’re looking at any one individual project, but in a large organization you’re almost certainly working on several different threads at the exact same time. That means that the need for effective tools and efficient processes may be even greater within these types of organizations than they are at a nimble startup. In a startup, you’re doing everything for the first time. You can’t be expected to have all of your processes ironed out, or make all of your graphs gorgeous. That can be exhilarating, and certainly very educational, but there is something to be said for the joy of working at near 100% efficiency. There’s nothing like craftsmanship with the sharpest tools.
On the other hand, startups have many fewer degrees of separation between the decision-makers, even at the highest level, and the analysts on the front lines. That leads to faster iterations between strategic decisions, new data, analysis and interpretation, and updated decisions. This means that projects may actually involve several analyses intertwined with action. This can be very exciting, but also quite chaotic and messy. Whether or not this sounds appealing may depend on your specific personality—the downside is that you can’t often get deep into any one analysis because it has to be turned around so quickly. In these situations, it’s the responsibility of a good manager to ensure that decisions are being backed up with facts and analysis, not gut feelings alone, as TextOre (an advanced text analytics technology firm) points out.
One of the biggest challenges of doing data in an enterprise environment is politics. If you’re doing honest analysis, for some portion of the time you will be delivering bad news. Even more complex, the news might be good for some people in the organization and bad for others. This creates a difficult position, particularly if you’re paid by the portion of the organization that is receiving the bad news. They should, if they’re perfectly rational, be glad that you’ve flagged an issue before it got even worse. But ignorance is bliss, and too many people react to a necessary data alert like Looney Toons react to an alarm clock.
In startups, it can be easier to dispense with the politics. You’re all on the same team trying to realize a common goal. However, even if your goals are clearly aligned, a small team may still find that their assumptions and therefore their preferred approaches can be quite different. Whether you call that politics or not, it can certainly lead to friction within the team. If your analyses become the scorecard on which the success of different people’s strategic decisions are judged, you can expect extensive scrutiny and backseat driving as you develop and report on your metrics. Even things that feel like they should be simple can be surprisingly hard to define in an agreed-upon way. For example, the salespeople might want to report their “Monthly Recurring Revenue Gained” without subtracting refunds, on the argument that those represent a product issue. But finance would never stand for that, because you can’t spend money that you’ve had to give back.
What can they learn from each other?
All this discussion of differences would be merely academic if these types of organizations didn’t have so much to teach each other. Many of the most successful corporations in the world have taken Clayton Christensen’s advice in The Innovator’s Dilemma, a book in which he argues that successful companies will not be able to ride the innovation that propelled them to their size indefinitely. They need to be prepared to disrupt their own business models, or else they will be disrupted from the outside. To put it another way, the next big thing must come from inside, or it will surely come from the outside (probably from a startup). Many companies are tackling this issue head-on by sponsoring internal startup teams that are free of corporate constraints. Google started down this path with GoogleX and then proceeded with a complete corporate transformation that made its core business just one of several divisions within a larger holding company.
Startups, too, can learn from larger companies. Although people complain about bureaucracy and red tape, there is an undeniable efficiency in having processes in place, so that each new situation does not need to be handled through trial and error. The trick is having the right amount of structure, so that you don’t miss innovative opportunities. But remember, time wasted reinventing the wheel on back-end workflows is time that is not spent innovating on core aspects of the startup’s value proposition, even if it’s not analysis paralysis. This is particularly true when it comes to data acquisition, storage and reporting. There are incredible tools out there for each of these tasks, many of which are designed to integrate seamlessly. Unless a startup is working with extremely big or unusual data, the right answer is probably not to develop a custom stack but to leverage tools like Hadoop and Tableau.
And, as Paul Carney (Deputy Managing Director of consumer insight agency Bonamy Finch) points out, there are ways in which businesses can get stuck, whether they are large or small. Lacking statistical rigor, even if they’ve nailed basic reporting, is certainly one. But even that rigor is useless without alignment with business goals. And none of that is going to help if a deep understanding of the customers is not baked into the process from the beginning.
The wave of innovation in the analytics space in the last few years has been truly amazing, and we’re lucky to be living in a time with so much opportunity in both the startup space and in the enterprise. The correct question is not “Which is better?” but “What can I learn from those around me to push myself, my business, and my industry forward?”