The State of Data Science & Analytics

arrow_back Epic Wins in Paid Search? Easy. Otis and the High-Stakes Rubiks Cube arrow_forward

by Allie Rogers

Recently, our team had the opportunity to attend “The State of Analytics,” a technology and data science event hosted by the Analytics & Big Data Society in uptown Charlotte. Additionally, John Thomas (who leads our data science team) was invited to speak about our paid search bidding platform, the Red Ventures project that won the North Carolina Technology Award for Best Use of Analytics in 2016. John explained the technology and algorithms behind that project, as well as the current state of data science and where the field is headed in the future. And we recapped his talk below:

The Current State of Data Science.

The days of automation and optimization are here, and we’re not just talking about physical assembly (like in automobile manufacturing), but in traditional analytical tasks, too. It’s also not just a matter of machines becoming cheaper than human employees – but that they’re becoming better in many ways.

Today’s machines can analyze data faster, more accurately, and in larger quantities than humans can. As a result, people’s jobs are changing. In the past, companies relied on data scientists to dig through data, find interesting nuggets, and report them back. Today, it’s becoming much more common to expect data science teams to deliver an impactful, optimized course of action. Placing impact at the front of the minds of data science teams will fundamentally alter the trajectory of the business and add significant value to the bottom line.

RV-Data-Science -John-Thomas

RV Vice President of Data Science John Thomas presenting at “The State of Analytics.”

Take paid search as an example. By bidding in a search auction, you’re paying for one bucket of potential volume that might come to your website. You can then track customers who go all the way through the sales process, determine which fraction of that traffic actually ends up buying, and thus have a very strong understanding of your ROI. Easy!

However, as your paid search efforts become more sophisticated, you could quickly find yourself bidding on thousands of keywords per day – and changing those bids based on time of day and locations of people searching. Red Ventures’ team of 40+ paid search analysts has done this with remarkable success for nearly a decade, but we wanted to see if machines could help us augment their performance even more.

Here’s what we did.

In order to make the most impact, we combined an algorithmic approach, technology, and business context. We started by looking for patterns in the quality of traffic we get. Our paid search bidding platform combines multiple models that eventually produce the optimal bids to maximize profit. The product team fully automated this process, and the tool is now making more bidding decisions per day than any human would ever be capable of doing.

Our solutions need to reflect the fact that the world changes. As new data come in, we need to:

  • Update our models
  • Generate bids from these models
  • Place bids on search engines and get data back from these search engines

This needs to become a virtuous cycle, so it was important to develop the right technology to automate these steps. We were able to do so thanks to our dedicated data science engineers. One of the most important developments by our engineers was the creation of a new R package containing dozens of functions specific to our platform.

This isn’t just an interesting exercise. Rather, it offers a scalable solution to a large-scale business problem. Since launch, this product has built more than 10 million models, placed more than 3 billion bids, and is on track to make a significant impact on the bottom line. Nice!

A few thoughts on the future of data science:

Its growth will be self-sustaining. Our paid search bidding platform is our story for how we got to the edge, but it’s not a unique problem. High-impact data science is achievable today, and it will spark self-sustaining growth for the field. As companies start to see that impact, they want more. This demand will merge business, data science, and technology closer together than ever before. It will push companies to weave data science into the fabric of what they do. It’s not inconceivable that many companies will entirely re-org around the idea of integrated data science.

There are a few companies, including Airbnb and Stitch Fix, who already do this. They’re the “early adopters” of integrated data science. These companies have already hired dozens of data scientists – and they’re continuing to find themselves with more demand for data science than they can hire into.

Specialization will be replaced by democratization. The shortage of highly trained data science talent coupled with the increased demand for data science will drive new entrants into the market. The new entrants will not only be newly trained data scientists, but also another important group: engineers armed with software tools. People will build software tools that bring companies more than 90% of the way to a full data science solution, democratizing data science and allowing for broader adoption of its practices.

When the supply of people equipped to do data science eventually outstrips the demand for data scientists, the artificial boundaries that some companies have built around their data scientists will also disappear in order to maximize impact. In fact, the CEO of RedHat was recently quoted in the Wall Street Journal forecasting that we’re at the cusp of a machine learning explosion in the software engineering space.

Companies that can out-race these industry trends have a huge opportunity to corner their markets by driving significant impact at reduced cost, allowing them to differentiate. There is an opportunity for everyone in analytics to challenge their companies to think differently about data science and the impact it can have on their business. We could each do this alone, but how do we produce a community of people that does this together?

Team Takeaways

John’s talk was well received and sparked even more energy around collaborating within the local data science community. Our team also learned about analytics industry trends such as virtual reality, predictive and prescriptive analytics, the Internet of Things (with Jen Underwood), and the neural mechanisms of object understanding (with Jiye Kim Park).

Since the ABDS event, our team has attended additional meetups and made plans to host a community event in the future. If we haven’t met you yet, we hope to see or hear from you soon!

Allie Rogers has been at Red Ventures since the spring of 2015. She recently moved from the operations team to the data science team after spending six months in RV’s first Data Science Accelerator Program. Outside of work, you can typically find her teaching yoga or jogging around South End with her dog.

Related Posts

The Red Ventures Guide to SXSW
RV Exclusive: Kenneth Cukier Predicts the Future of Data Science
Building a Great Engineering Culture