Data-Driven Product Management at Groupon

Data-Driven Product Management at Groupon

Data-Driven Product Management at Groupon

Laura Hamilton
Group Product Manager
May 22, 2018

Introduction

At Groupon, we have a very data-driven philosophy of product management. In this blog post, I’ll talk through how we approach product ownership in a data-driven way, from financial forecasting to roadmapping to feature development to experimentation.

Hippo

https://www.howgoogleworks.net/

Financial Forecasting and Roadmap Creation

For every candidate feature, we calculate the projected financial upside according to the following formula:

feature_revenue_forecast =
expected_lift x platform_factor x success_probability x platform_revenue

Where:

  • feature_revenue_forecast is what we are trying to calculate (the expected revenue from the feature)
  • expected_lift is the increase in conversions we expect from users in the treatment group vs. users in the control group. The vast majority of experiments fall between -1.0% and +1.0% lift.
  • platform_factor is what percent of all users of the platform (whether iOS, android, mobile web, or desktop web) are part of the experiment. For example, for a test on the checkout page on mobile web, the platform factor will be 100%—100% of users who place an order on mobile web visit the checkout page during their journey. For a test on the getaways deal page, the platform factor will be much smaller, so the overall financial impact of the test will be smaller.
  • success_probability is a haircut we apply to take into account that not all experiments will succeed. In fact, only about 30% of our experiments are successful. The success_probability for a given feature could be greater than or lower than 30%, depending on how confident we are that the experiment will succeed. For some experiments that are primarily for strategic reasons, such as maps improvements, we will use a success_probability of 90% or 100%.
  • platform_revenue is the total revenue generated by the platform. For example, the platform_revenue for iOS is the total revenue from orders placed via the iOS app.

With this formula, we have a consistent and data-driven way to estimate the upside from each proposed initiative. Then, once each experiment concludes, we compare our estimates to the actual results, and over time we refine our estimations.

We use these estimated upside figures to create product roadmaps. In order to prioritize initiatives and determine the cutlist, we need to introduce another data point—the engineering effort required. Then, we use the following formula to calculate the ROI of each feature:

ROI = feature_revenue_forecast / level_of_effort

We then stack rank features according to their ROI.

ROI is an input into the creation of the product roadmap and the determination of the cutlist, but it is not the only input. I always like to ensure that there is a healthy amount of time spent on engineering excellence (site stability, paying down technical debt, reducing latency, library upgrades, increased test coverage, improved tooling). I also like to ensure that we have a customer focus. Many of our features come directly from customer feedback via focus groups, quantitative surveys, and app store feedback; the Wishlist feature was one of these. I also reserve a healthy amount of time for strategic initiatives that may not provide lift in the short term but that set us up for success in terms of the Groupon 2020 vision.

Dilbert

Image source: Dilbert.com

Data-Driven Features

At Groupon we are lucky to have vast amounts of data that we can use to deliver a delightful product to our customers. We have worked with one million merchants to date; we have pumped more than $18 billion into local businesses; we have more than 1 billion Groupons sold; our app has been downloaded 171 million times, and we have saved customers more than $28 billion.

The Groupon platform handles tens of billions of user actions per month, and for machine learning algorithms that drive core product features our platform needs to make decisions (such as which deal to show the user next) in fractions of a second.

Developing product features that take advantage of these vast amounts of data in a performant way is an interesting challenge.

We use machine learning algorithms in a variety of ways to develop products here at Groupon:

  • Supply intelligence – There are millions of merchants we could call at any time to get onto our platform; how do we pick the best ones?
  • Fraud prevention – Fighting the bad guys in realtime.
  • Discovery and personalization – Selecting which deals to show a given user in her mobile app deal feed.
  • Image recognition – Identifying the best user-generated images with neural networks.
  • Logistics – Getting ahead of the order rush by sending extra inventory to the right warehouse in advance of high demand.
  • Customer support – AI-based chatbots to respond to and resolve customer issues instantaneously.

Groupon Mobile App

To make developing data-driven products faster, we built a generic, extensible machine learning platform at Groupon called Flux. Flux is the “Rosetta Stone” between data scientists and engineers.

Flux capaciter

Image: Wikimedia Commons

Data scientists work primarily in R. Flux models are written in Java and Clojure for stability and speed. Python is the glue that connects R and Java. It all runs on Groupon’s large Hadoop cluster.

To make the process for productionalizing machine learnings more robust, Groupon has an ETL management platform called Quantum Engineered Data (QED). QED reads from any source, and includes built-in data cleaning, error correction, and anomaly detection. Clean data is preserved and made available as a “feature catalog.” QED handles failures smartly, supporting falling back to yesterday’s model when appropriate. QED is able to plug into any source of truth—including streams, warehouse tables, and JSON endpoints.

Smart people doing interesting work

QED gives us a lot more confidence in the robustness of our models. In general, subtle changes to a single data field can seriously impact model performance, and nuances in the data set could look fine to tests but fail in the real world.

machine learning - xkcd

Image credit: XKCD

Monitoring

This blog post would be incomplete without a brief discussion of Groupon’s monitoring tools. We have a healthy suite of realtime alerts on product and engineering KPIs. We use splunk for logging and wavefront for graphing. Each service is staffed with a 24/7 on-call schedule, with escalation handled by pagerduty.

Additionally, each product area and business has an Amazon-style Weekly Business Review, where we look at metric trends longitudinally, identify areas of change or concern, and begin deep dives where appropriate.

The data warehouse uses Teradata and Apache Hive.

Experimentation

There are 100 teams at Groupon that run experiments. At any given time, around 200 experiments are being run simultaneously on the Groupon platform.

Groupon has a dedicated team called Optimize that built a bespoke tech platform for running product experiments with mathematical rigor. The experimentation platform is called Finch Express. Finch Express is built with Ruby on Rails, Node.js, Ember.js, Python, R, and Hadoop/Hive. The team has filed three patents for its innovations on product experimentation.

Essentially, Finch Express uses a technique called Group Sequential Analysis, first developed by Abraham Wald in 1945. Group Sequential Analysis has been used extensively in high-risk clinical trials, such as heart valve studies, where it’s possible that one treatment is actually harming the patients. Ethically, we would want to stop a harmful clinical trial immediately—but statistically, checking the experiment results mid-run or “peeking” will vastly increase your rate of false positives and invalidate your statistical results.

Group Sequential Analysis provides a controlled, statistically rigorous way to “peek” at experiment results at set points during the experiment run. This allows Groupon to end an experiment early if it is losing money, and to roll out an experiment early if it is deemed an early winner (capturing more upside).

Finch Express does all of this automatically. Product managers create the experiment in Finch Express, add a description and screenshots (to save the details for future product managers to reference), and launch the experiment at 50/50. Finch Express does the heavy lifting of dynamically determining the appropriate lift sensitivity for the experiment (based on traffic and conversion rate), performing the Group Sequential Analysis calculations, deeming the experiment a “success,” “failure,” or “flat” (most experiments end flat), and even automatically rolling out or rolling back the experiment based on its results. Then, Finch Express reports on the financial results of the experiment. The experimentation platform prevents product managers from statistical no-nos, such as peeking, unbalanced bucketing, and concluding the experiment too early. As a result, our experimentation processes have a high degree of statistical rigor.

On average, Group Sequential Analysis allows us to conclude experiments an average of 57.53% earlier compared to simply running them to a single final checkpoint. This reduces the cost of failed experiments, hastens upside capture of successful experiments, and allows for much faster iteration and innovation.

To date, Groupon product managers have run a total of 2,500 experiments, thanks in large part to the proprietary and patent-pending experimentation platform.

correlation - xkcd

Image credit: XKCD

Conclusion

Thanks for staying with me until the end! I hope this gives you an idea for how we use big data at global scale here at Groupon to create our product roadmaps, to innovate with new products and features, to monitor product performance, and to evaluate the impact of new initiatives. If you’re interested in learning more about data-driven product management opportunities at Groupon, have a look at our open roles.

Laura heads up product for consumer web, international, and LivingSocial at Groupon. She has a bachelor’s in mathematics from the University of Chicago and a master’s in computer science with specialization in machine learning from Georgia Tech. She has more than 10 years of experience in ecommerce product management at four Chicago tech companies, from early stage startups to publicly traded global companies. She is passionate about using analytics and machine learning to create a delightful customer experience.

Laura Hamilton

Group Product Manager

Laura Hamilton – Group Product Manager

Laura Hamilton – Group Product Manager

LAURA HAMILTON

Preferred pronouns: She/Her
Group Product Manager
Chicago, IL
Joined: 2014

Laura and Groupon seem to be a perfect match. She brought 10+ years of strong product management experience to Groupon. In turn, Groupon provided an opportunity to get scrappy, make an impact, and walk by a cat spaceship every day. (What else do you really need?)

What was your first role here? How did you get to where you are now?

I started off as product manager for the checkout page, and I gradually took on more responsibility. I took on Consumer Web in Q1 2016, LivingSocial in Q4 2016, and International in Q1 2017. Today, I head up product for Consumer Web, International, and LivingSocial.

Where were you before joining Groupon? How did you get here?

I have worked in product management for 10+ years, at a wide range of company sizes from founder to seed-stage startup to hypergrowth phase to a public company. I started my product management career at Enova 10+ years ago, where I led the product team focused on new initiatives. We launched several new businesses and product lines during my tenure, including the installment product in the US and the Pounds to Pocket brand in the UK. After Enova, I worked at a couple of early-stage tech startups. Then I joined Groupon in 2014 — Groupon still has the scrappiness, innovation, and bias for action typical of startups, while also having massive global scale.

How does your work connect to Groupon’s mission of being the daily habit in local commerce?

I always try to tie my product strategy and roadmap to Groupon’s mission of being the daily habit in local commerce. Whether it’s adding more structured data to the cards on the browse feed, making our pricing clearer and more transparent, launching new products such as Groupon+ and BeautyNow, it’s all in support of our mission to connect customers with local businesses.

What’s the most challenging thing you’ve worked on here?

In 2016, when I took over consumer web product, our mobile website was very outdated and lacked many key features that the mobile app had. Over the course of 2016, I led a major effort to redesign the mobile website, and we overhauled almost every page in the site. It was a big undertaking that involved a dozen different teams, but after several months and a few dozen A/B tests, we had a much more organized and modern website.

Take us back to 2014. What was your first impression of Groupon?

I think I was going to the Geekfest meetup for the first time, and the instructions said to look for the “cat spaceship.” I was very confused about what a cat spaceship was, and why it would be in an office. I was somewhat nervous that I would miss this landmark; however, upon seeing the cat spaceship, I realized that my fears were unfounded.

What’s something that has surprised you while working at Groupon?

Groupon’s Optimize team, which owns the experimentation framework and methodology, has a novel (patent-pending) process for the analysis of A/B experiments. It uses a technique called group sequential analysis, which originated with heart valve studies in the 1970s. It’s essentially a statistically rigorous way to “peek” at experiment data while limiting false positives. It has allowed us to experiment much faster, which in turn has led to faster improvements in the customer experience.

So what’s unique about Groupon’s Product culture?

Groupon is unique in that Product Managers can take on a ton of responsibility—whole product lines, or product suites—while still operating at massive, global scale. At startups, you can have broad responsibility but not at scale; at most publicly traded companies, you can have global scale but you only own a small piece. At Groupon, product managers can have broad responsibility as well as massive impact at scale.

“Groupon is unique in that Product Managers can take on a ton of responsibility — whole product lines, or product suites — while still operating at massive, global scale.”

What’s your favorite part about your team’s Software Development process?

Software development at Groupon is very solution-oriented. If there’s a bug or production issue, the discussion is immediately around how to solve the issue and how to prevent it in the future. We don’t have unproductive fingerpointing conversations. When there is a production outage, the discussion is around “how do we improve tools and processes to prevent this from happening in the future,” rather than “so and so made a mistake.”

What’s your favorite Groupon memory?

After six months of work across dozens of engineering teams, we successfully migrated the LivingSocial website and mobile apps to the Groupon tech stack and celebrated with a cutover cake and champagne at 3 am.

Name your favorite programming language.

I think it depends on the problem domain, but in general, I am partial to python. The syntax is natural. I don’t need to define variable types or worry about semicolons. Also, python has a very thorough set of supporting libraries, including a lot of machine learning and analytics related libraries.

Tabs or spaces?

Spaces

Vim or emacs?

Vim

What do your parents think you do?

I asked my parents what they thought I did, and their answers were actually pretty close. Both my parents have PhDs in economics. My mother said, “I think you handle the website. I would love to know.” My father said, “Management/analysis.”

What keeps you busy outside of work?

I love checking out new local restaurants and fitness studios. I live right by Restaurant Row, and there are so many fantastic local businesses in my area. There are new ones popping up all the time — it’s hard for me to keep track!

Speaking of local restaurants, what’s your favorite local business?

It’s so hard to pick just one. I love Big & Little’s; their gourmet fast food is so delicious.

Tell us something that might surprise us about you.

I have done two startups – neither of them successful.

Joanne Soo – Senior Technical Product Manager in San Francisco

Joanne Soo – Senior Technical Product Manager in San Francisco

JOANNE SOO

Senior Technical Product Manager, Groupon+
San Francisco, CA
Joined: 2011

At a young age, Joanne was told she wouldn’t be able to do any overly physical activities due to her disability. Fast forward to today, she’s not only climbing three-story walls but is also applying her fierce determination to a growing Product role at Groupon.

Tell us about what you do and what you love about it!

I work on the Groupon+ Product team, focusing on consumer growth. I love that I get to launch really cool features, learn what our customers love and what they don’t, and collaborate with the best and smartest people in the industry to shape the way people save money while dining out.

What’s the most challenging thing you’ve worked on here?

Back in 2015, while on the Conversion team, I’ve had the opportunity to drive the North America PayPal integration, which was a feature that was very high in demand from our consumers. This effort was particularly challenging because the integration happened during a time where Groupon was rolling out a set of new technologies (new API endpoints, new backend services, and a few ongoing frontend UX A/B experiments). The challenge was to launch this initiative on top of those rollouts without compromising on performance and user experience. There was a lot of careful planning needed and strong coordination between various teams proved to be very valuable. We were able to successfully launch the payment method and it was a big win for customers and for Groupon, which was really, really awesome!

What’s unique about Groupon’s Engineering culture?

I love the diversity and our continuous pursuit of a diverse workplace. One of our core values (Groupon’s) is “respect, integrity, and inclusion” and while this is important at any workplace, it is even more so in engineering departments, which are usually predominantly masculine. At Groupon’s Engineering organization, our proactive fostering of a diverse workplace has created a place where people feel comfortable challenging each other’s ideas, regardless of where you come from, your gender, sexuality, identity, etc. Also, cats. What’s not to love? 🙂

And what specifically do you enjoy about Product at Groupon?

I really enjoy being able to launch really cool features. As owners of our specific areas, we have a direct influence on the outcome of the product, and through that, we get to learn about consumer behavior, our merchants, and Groupon as a business. And because we operate in a lean, agile environment, we get to launch, learn, and iterate quickly and see our product evolve over a short span of time.

So what did you do before joining Groupon?

I worked for a startup in Malaysia that focused on youth empowerment. One fine day in 2010, we decided to pivot our business model and try something new. We built GroupsMore.com, a Groupon clone offering deal of the day to Malaysians. Within four months of launching, our company was acquired by Groupon. Since then, I’ve worn many hats at Groupon (customer service, editorial, project management, and now product) and have lived in various countries for the job: Malaysia, Taiwan, and now the US.

What’s your favorite Groupon memory?

In Groupon Malaysia, we had set up an RM9,000,000 unicorn ride deal ready for April Fool’s Day in 2012. We decided not to launch it at that time and we killed the deal before we launched it (but the deal URL was still live, and accessible if someone had access to it). Fast forward to February 2013— something mysterious and weird happened—we saw a huge spike in traffic on groupon.my and found that the (closed) unicorn ride deal had for some reason gone viral! We were worried about bad press at first, but the virality actually was a good one…people were amused. Here’s an article about it. Needless to say, sales skyrocketed that week. 🙂

What keeps you busy outside of work?

I climb walls! I was born with a disability and was told as a child growing up that I would not be able to do any big physical activities, but with my nature of breaking molds + wanting to prove people wrong + in the spirit of “I can do it even if you say I cannot!”, I’ve developed a passion for scaling three-story walls at climbing gyms.

Tell us something about you that might surprise us!

I have synesthesia, a neurological phenomenon where one’s senses are intertwined. Some people taste words (lexical-gustatory synesthesia) or feel sounds (auditory-tactile synesthesia). I see numbers and letters in colors (grapheme-color synesthesia), which make memorizing/remembering spelling, names, strings of numbers relatively easy for me.

What do your parents think you do?

My mom thinks “I do all the creative planning for the internet and website things” and my dad thinks “something with people management skills or something.”

Ana Ananthakumar – Product Manager, Supply Intelligence

Ana Ananthakumar – Product Manager, Supply Intelligence

ANA ANANTHAKUMAR

Product Manager, Supply Intelligence
Chicago, IL
Joined: 2017

Ana describes his job as being an agent for Groupon’s version of the CIA. What does that entail? We’d tell you, but…just read his story to find out!

What’s your favorite Groupon memory?

On the day before Halloween, I was busy at my desk working and wearing a T-shirt and jeans (on that day, my tee-shirt and jeans combo was the most formal attire). I was already enjoying the costumes people around me were wearing. Top Gun’s Tom Cruise (our engineering manager), was sitting with his costume. A few desks away, a fox was coding. I thought that things couldn’t get any more hilarious. As if to prove me wrong, I saw all hell break loose in our office: Freddy Krueger, Jason, and a lot more of the everyday villains that haunted you were walking among us.

Where were you before joining Groupon?

I was born and raised in Sri Lanka. I graduated an Electrical Engineering major, completed CFA Level 3,  and worked for Unilever’s supply chain for four years. Then, I came to Kellogg to get my MBA. This was the first time I lived outside of Sri Lanka. Last summer, I was the product manager for a drone company. Now I am here!

So tell us about your job. What do you do? What do you love about it?

I am the Product Manager for Supply Intelligence. My product—an AI platform—brings in the best supply/merchants into Groupon. I think of it as GIA (Groupon’s Intelligence Agency – like CIA). Think of it as an ‘agent’ going out and looking at all the available merchants in the market, and gathering intel about which merchant would maximize Groupon’s value in the future ⇒ repeat. I love the fact that this is at the cutting edge of technology, and that I get to impact the entire Groupon, both North American and international as well. (This is the intimidating component as well!)

Where can we find you outside of work?

I am on okay cook (Sri Lankan and South Indian cuisines) and an excellent eater. I have won eating competitions (26 pieces of pizza in one instance). I used to play rugby and chess for school/college. I am a singer and a disaster of a dancer. Currently looking for friends who might go to fighting classes, dancing (you’ve already been warned), or climbing with me.

Tell us something about you that might surprise us.

I eat a lot of sweets. One of our team members brought in a bag full of candies that were too much even for his kid (remember, this was right after Halloween, so the bag was really big). I have been constantly eating away all those candies, alone – without the slightest hint of guilt.

I always believed that with the agile methodology things would be fast paced, but at Groupon, I have seen some of the ideas go from concept to implementation within a few sprints.

Tell us about your first impressions of Groupon.

My first interactions with Groupon were with the recruiter, an alum, and the hiring manager. I got the sense that Groupon had a very open and inclusive culture because all of these folks focused on genuinely listening to my story and understanding what I wanted to do. I also felt that Groupon focused more on potential when it came to recruiting, rather than experience (slope vs intersection argument: Groupon hires more so for the slope). This means that not only does Groupon benefit from its employees, but it also has a keen interest in investing in their skills and career development.

What do your parents think you do?

They still can’t wrap their head around the fact that a tech laggard like me is handling a product that involves a lot of AI. “So your product has something to do with AI right? And you still take notes on paper? Son, you’re like from the stone age.”

What’s unique about Groupon’s Engineering culture?

How collaborative and organized the team is. In my team, I could find a textbook example of the Agile process, and every aspect of it. We have an excellent scrum master who works on the finer details of the agile process and drives it like a religion. I remember the entire team instinctively getting up and gathering around his table at 10:30 for the daily stand…even on Mondays, the only day of the week we don’t have that meeting. As the book, “Scrum”, says, he is driving the agile methodology into our bloodstream, as a way of work that would come naturally to us.

What about the Product culture?

Despite being a large organization, I can directly see how my product impacts the business. I have responsibilities that create a meaningful impact on the entire business. This also means that I should be extremely careful about the changes I make to the product given the level of impact it has on Groupon.

What has been surprising about working at Groupon? Or what would surprise candidates to find out?

Surprising to me: How fast-paced things work. I always believed that with the agile methodology things would be fast paced, but at Groupon, I have seen some of the ideas go from concept to implementation within a few sprints.

For external folks: Work hard, play hard culture. I’ve been at Groupon for a few months and already have been with various teams to dinners, improv shows, breweries, and even rock climbing. It is great to see that the teams truly enjoy being together whether at work or outside of work!

What’s your favorite view from the office?

How does your work connect to Groupon’s mission of building the daily habit in local commerce?

My platform brings in the local commerce to the sales reps. Very direct connection.

What’s your favorite conference room?

What are your favorite local businesses?

Horror houses. I visited one and learned that Groupon has a big impact on the incoming traffic to the industry during the Halloween season.

What do you love about your team’s Software Development process?

How well defined the user stories are and how articulate the engineers are about the user stories. The dev team has a good ‘big picture view’ of how some of the changes might have downstream and upstream impact on the business. This makes it easy for me to connect the dev process with the other parts of the business and the respective stakeholders.

One of our software developers created a presentation which illustrated our process end to end. I use it all the time when I present to others about my product and process. Not only was it easy to understand, it was also quite comprehensive. Furthermore, we have a platform called Flux for our machine-learning models. As you might have already figured out, it’s a very hard concept to grasp. The engineer in charge of that explained it to me in such a manner that I could well understand what it meant for my product and for Groupon in general.

We have another engineer who has an amazing curiosity to learn. He makes sure to ask questions to hammer out the details. He sweats the details in meetings and finally when the products are built, there is no question as to any of the nuances being overlooked. I believe that this ability to articulate so well and curiosity, are key ingredients to a successful process.

John Stokvis – Technical Product Manager

John Stokvis – Technical Product Manager

JOHN STOKVIS

Technical Product Manager
Chicago, IL
Joined: 2011

What’s your favorite conference room?

It’s not every day you get to interview a Product Manager whose secret talent is being able to balance 3 chairs on their chin…because they were once a circus performer. Talk about making big career moves!

What’s your favorite Groupon memory?

My favorite Groupon memory isn’t something big or grand. It was actually a small act, but it’s resonated with me ever since. It exemplifies what I think is special about Groupon and is the source of a lot of our success. Around 2013, I was working as an analyst building dashboards. Unprompted, a more senior employee gathered a group together (analysts and non-analysts) for a tutorial on SQL. He created a whole powerpoint presentation and walked us through concepts and small tricks he had picked up. He even scheduled several follow-ups with the group and was always available to answer questions and learn tricks. He wasn’t even on the same team as I was, but saw an opportunity to help us all improve. He’s no longer at Groupon, but I constantly think back to what he did for me. I try to keep that part of the culture alive. Supporting your teammates and making each other look good is pervasive at Groupon. It’s what I love most about working here.

So before joining Groupon you were…

An actor and circus performer. I did some theater and circus around town, as well as the occasional temp job. An actor in one of the shows I was in helped me get a job in customer service and I’ve been here ever since.

If you can remember back to 2011…what was your first impression of Groupon?

I remember it flashing across my consciousness before I started working here. It was this super cool way to get a great deal with your friends. And it just showed up in your email inbox! It’s been crazy to see where we’ve come from there.

You have an interesting career path here…tell us about it!

I started as a front-line Customer Service Rep, helping folks find their Groupons, cancel purchases, and solve issues with merchants. We had a ping-pong table in the office and there were about 10 of us who got pretty good. We all started talking about who was the best and on my breaks, I built a spreadsheet in Google Docs to figure it out (I Googled a bunch of formulas and read about the ELO ranking system – the same one that Nate Silver uses a lot on FiveThirtyEight). Then I got people to enter the results of their matches to determine who was the champion (hint: it wasn’t me). About a year in, there was a special project to use predictive analytics to predict which deals would be likely to have lots of refunds. Because of the ping pong spreadsheet, they gave me a shot and I loved it! I learned all about SQL and analytics, ended up joining the fledgling Risk Management team as their first analyst (there were 4 of us at the time, now it’s a global department!).

From there I moved to Global Operations to help build dashboards and analytics for our efforts to take Groupon’s global footprint and streamline it; at that time, we were operating in almost 50 countries, sometimes it seemed like 50 different companies, all called “Groupon.” After that, I learned about Product and moved over here. I worked on our backend merchant database for about a year and half. When Groupon acquired LivingSocial in late 2016, I joined the small core team to lead the Product effort to rebuild LivingSocial on Groupon’s infrastructure. We launched our MVP in June 2017, and I’ve been working on improving LivingSocial and the multi-brand platform ever since!

What do your parents think you do?

I asked my parents and they got it pretty spot on. My theater friends always give me blank stares when I tell them I’m a PM. So I just tell them I’m like the director and the developers are like the actors – they’re real stars. I just put myself in the mind of the audience (the customer) and point the engineers in the right direction. Then my metaphor breaks down and they nod and change the subject.

What’s unique about Groupon’s Engineering culture?

We’re scrappy. A lot of people forget that Groupon’s origin was as a hack on top of The Point, which was like a proto-Kickstarter. The very first Groupon was a deal for 2 pizzas for the price of 1 at the bar downstairs at our office, but people could only get it if enough people “pledged” to buy it. That sense of “let’s try this and see if it works” and then iterating on that is part of our DNA.

The local commerce or online-to-offline (or whatever the kids are calling it these days) is really tricky. What works for one kind of business doesn’t work for others. Everyone in Product and Engineering is game to try new things, then take the feedback from customers and merchants and learn from it. I think that’s one of the main reasons Groupon has stuck around while others haven’t. Even large companies like Google, Amazon, and Facebook have tried to get into this space and haven’t been able to crack it.

And more specific to you, what’s unique about Product Management at Groupon?

The collaboration. A lot of people think of PMs as “mini-CEOs” and some companies have PM cultures that are cut-throat and every-PM-for-her/himself. At Groupon we’re constantly sharing thoughts, partnering with each other on projects, and bouncing ideas off each other. Not all PMs come from a product background. Some do, but many come from Customer Service, Sales, Account Management. There’s a variety of experiences and backgrounds that we collectively draw from and a lot of cross-pollination.

What’s your favorite programming language?

Ruby on Rails. Because it sounds like the name of a small town in northern England.

What’s the most challenging thing you’ve worked on at Groupon?

That would be the project to rebuild LivingSocial on the Groupon platform, what we referred to internally as “Project Gemini.” Looking back, it was a truly massive undertaking. We were modifying everything, from backend to frontend, how customer service took incoming calls to how we processed payments. It touched five different departments, impacted more 300 different individuals directly, had more than 60 different workstreams. The core team that I was a part of had to not only keep track of all this action but see how the decisions we made in one place would affect things somewhere else and make the right call. It was super challenging and when we ultimately finished our MVP and cutover from old to new LivingSocial and the 6 months of work that everyone did finally pay off, it was an amazing payoff. Someone even brought a “cutover cake” that we ate at 3 am with some champagne. And that was just the MVP – getting us to the starting line!

Tell us about what your job entails and what you love about what you do.

I’m the Product Manager for LivingSocial, so I focus on the modifying and adapting the Groupon platform for the LivingSocial brand. I work closely with our Engineering, Operations, and Marketing teams to make sure the customer experience is as great as it can be.

My favorite part of the role is simply the breadth that I get to be responsible for. Most teams have a specific area they focus on (the deal page, redemption, or a single vertical like Goods or Things To Do). With LivingSocial, the team gets our hands in every part of the site. This means we often have to work together with other teams to come up with solutions. Every day I get to solve a huge variety of interesting problems, so it’s never boring. LivingSocial is the first time Groupon has tried to run a second brand on the platform, we get to tackle all sorts of new problems that the company has never had to solve before. On top of it all, I’m getting a big picture sense of how the whole system works, I get to see all the connections, which is super cool.

Tell us something about you that might surprise us.

I can balance three chairs on my chin.

What keeps you busy outside of work?

I do the occasional on-camera acting gig. My wife is an actor and I’ve still got friends in the Chicago theater scene, so I get to see a lot of really interesting theater. Chicago’s also a great biking city so I try to bike to work as much as I can (it also helps feed my podcast obsession). I love to bake (my favorite is sourdough bread) and we keep chickens in our backyard so we get fresh eggs (no, we won’t eat them and yes, they’re fine with the Chicago winters). Fresh eggs and toast definitely make breakfast my favorite meal of the day.

What’s your favorite local business?

Antica Pizzeria in Andersonville. Amazing Sicilian pizza (thin crust, not “Chicago style”) fresh out of a brick oven. The best crust in the city. My favorite one is arugula, pistachio, and speck (kind of like prosciutto) and a drizzle of olive oil. It’s delicious!

What is your favorite part of your team’s software development process?

Our retros. We really dive into the task of constantly optimizing our process. Talking about what worked and what didn’t. Why issues happened and what we can do to avoid them in the future. We try to keep a sense of psychological safety so we can be honest with each other.

Show us your favorite view from the Chicago office!

How does your work impact Groupon’s mission of building the daily habit in local commerce?

With two brands in the portfolio now, my job is to think about what LivingSocial customers are looking for when they visit the site and help them find it. Groupon and LivingSocial were competitors for most their existence but now they can work together to help customers explore the amazing things to do and see in their neighborhoods.

Tabs or spaces?

Tabs all day long.

Vim or emacs? Why?

Andrew Lisy – Director of Product, Marketplace

Andrew Lisy – Director of Product, Marketplace

ANDREW LISY

Director of Product – Marketplace Team
Chicago, IL
Joined: 2014

From bond trading on Wall Street to e-commerce in Chicago, Andrew has taken his high-stakes modeling experience to the more local-focused (and probably slightly less stressful) environment of Groupon.

Tell us about what you do! What do you love about it?

I lead our marketplace teams, which comprise everything related to supply and pricing. In the supply world, we split it into two groups — Supply Intelligence (SI) and Scalable Supply (SS). The SI role is really fun because it’s heavily driven by machine learning models and decision science. I used to be a Wall Street bond trader, so a lot of the mechanics that go into determining the best merchants to bring onto Groupon’s platform are very, very similar to the models I used to use to pick the best stocks and bonds to trade as a trader. On the scalable supply side, I get very involved in partnerships with a variety of companies that do really cool stuff in the local space. We work with these companies to list their inventory on our platform. Finally, pricing is really fun for a lot of reasons — there’s the fun/academic/nerdy/trader/financial side of building out elasticity models, the immediate gratification of seeing our work appear on the front page of the site, and the thrill of working with an amazing engineering and data science team.

What do your parents think you do?

Me texting my mom: “What do you think I do for work”

Mom: “Big data? Head of all data? Analyzes data? Did you get a sitter for Friday?”

So what’s unique about being a Product Manager at Groupon?

More than anywhere else I’ve worked, being a PM at Groupon means ownership. PMs really own their areas and are ultimately responsible for the success or failure of their product(s). It’s an incredible feeling — every day, I come in and influence a big part of a major e-commerce site with tens of millions of users.

And how does your work impact Groupon’s mission of building the daily habit in local commerce?

Incredibly tightly! In order to create a daily habit, we have to have amazing supply and incredible prices. The work I do every day ensures that the merchants we’re bringing onto Groupon are the best ones to engage and delight our customers when they pull the app out of their pocket. If I do my job well, Groupon becomes an indispensable place to check every day to find great merchants around your area.

In order to create a daily habit, we have to have amazing supply and incredible prices. The work I do every day ensures that the merchants we’re bringing onto Groupon are the best ones to engage and delight our customers when they pull the app out of their pocket.

What’s your favorite Groupon memory?

When we launched card linked offers (CLO), we got a group of product managers in Chicago to go bar hopping to 3 or 4 bars that were in CLO. It was grueling work, and we were all in pretty rough shape after several hours of exhaustive product testing (and testing. and more testing). We try to do this sort of thing every time we launch something new. It’s a really fun way to “eat our own dogfood” and have a great time with co-workers.

Where were you before joining Groupon?

I was a trader — I traded convertible bonds at Merrill Lynch, options at a hedge fund, and then worked in strategy at a high-frequency trading firm. I loved the excitement of the markets, but something really clicked when I started building my own trading platforms for me and my trading group to use. One day, I work up and realized that building the platforms was what really thrilled me, and the trading part was just incidental. At that moment, I decided that I wanted to move to Product.

Smart people doing interesting work

 

What was your first role here and how did you get to where you are today?

I started as a Senior Product Manager on a product called Quantum Lead (QL). QL was (and is) a very data intensive program and tightly connected with our sales org, so I spent a lot of time with our data science teams and sales teams. After working on that for a while, another fairly quantitative role opened up leading our pricing team, and my experience was a good fit to take that on as well. We’re always looking to give people new challenges and opportunities, and it was a really fun space to dive into.

Where can we find you outside of work?

I like making stuff, toys, and eating. Cooking satisfies all three of these interests, so I’m really into making delicious food. I’ve also been getting into woodworking lately because it’s fun to come home from a long day of making code and jump into making physical stuff out of wood. I like making cutting boards because they’re fairly complicated to make but also can come out looking really nice. Finally, I like photography, so I document both of these hobbies on some small personal blogs I keep around.

And lastly, Vim or Emacs?

nano (/nerdcred). I used to be a pretty heavy emacs user but basically lost all of that muscle memory.

Data Driven Chicago, Full Video Recap

Data Driven Chicago, Full Video Recap

ENGINEERING & MACHINE LEARNING

Data Driven Chicago has its second data industry event with Echo Global Logistics, Reverb, Sprout Social and Trunk Club

7 November 2017

Complete recording of Data Driven Chicago, November 2, 2017

Recorded at Groupon HQ in Chicago

If you missed it, two of Groupon’s own were joined by folks from Echo Global Logistics, Reverb, Sprout Social and Trunk Club to talk about the current challenges and innovations in the data-driven product  and machine-learning space. This event featured presentations by:

  • Ilhan Kolko (Echo)
  • Andrew Lisy and Laura Hamilton (Groupon)
  • Tyler Hanson (Reverb)
  • Mary Feigenbutz and Greg Reda (Sprout Social)
  • Laurie Skelly and Elizabeth Cleveland (Trunk Club)
  • Moderated by Alli Diedrick (Built In Chicago)

Groupon’s project managers are ‘the air traffic controllers of the engineering department’

Groupon’s project managers are ‘the air traffic controllers of the engineering department’

PROJECT MANAGEMENT & ENGINEERING

Groupon’s project managers are ‘the air traffic controllers of the engineering department’

30 October 2017

The Groupon Project Management Team in Chicago

Photograph by Chris Murphy

It was 4 a.m. when confirmation finally came in from the engineers: Groupon had successfully completed the online integration of LivingSocial, which the company acquired in 2016.

Groupon’s project management team had been toiling for months, working with engineers and key stakeholders on this large-scale project. With about 50 other staffers from different departments, they waited and watched into the wee hours, some via conference, until the “cutover” was finally confirmed. This means that users could now access the integrated content — and two weeks ahead of schedule.

That’s when an engineer came in with a bottle of Cristal and a “cutover” cannoli cake.

“Besides my wedding cake, that was the best cake I’ve ever had,” said Karen Hyatt, technical project manager. “There was something about being bleary-eyed at 4 a.m., eating cutover cake with the team you worked so hard with over six months that was really just special.”

Read the complete article at Built In Chicago.

Senior Getaways Product Lead in Seattle

Senior Getaways Product Lead in Seattle

Groupon Getaways is the travel arm of our business, connecting consumers with hotels, BnB’s, and resorts across the globe. Today, Noah Spitzer-Williams chats about how much impact Engineers have on the business and also sheds light on the quirkier side of Groupon.

So what do you do? 

I lead the Consumer Experiences team for Getaways, Groupon’s travel business. We sell almost $1B worth of hotel and tour package deals a year. My team obsesses about building great experiences for travelers in a dozen countries across both our web and mobile platforms. We’re “full stack” in that we build everything from the user interfaces to the backend services.

My favorite thing about product management at Groupon is how much exposure we get to all elements of the business. In a typical week we’ll work with folks in engineering, design, business development, marketing, customer support, and analytics. We have impact on everything from the “why and what” to the “how and when.”

What did you do before Groupon? 

Immediately before Groupon, I ran an action sports video editing startup. We weren’t financially successful but the experience was phenomenal. I learned how much hustle it takes to make it on your own. The Cinderella stories of successful founders you read in the news are not flukes. I also learned that I love building products much more than dealing with finances and legal documents.

What was it that made you want to join Groupon? 

Long story short, I had already done both the big company and the startup. I wanted something in between that had the resources of a big company but the entrepreneurial attitude of a startup. I also wanted to work in travel. Getaways was the perfect fit. It was a small team, but they were hungry and had big ambitions. Groupon itself was only 4 years old yet it was making billions all over the world. In effect, Getaways was a startup within a startup.

What’s unique about the work Groupon Engineers do?

There aren’t many companies that provide engineers with as much opportunity for impact as Groupon. Engineers here work side-by-side with product managers, designers, and folks from the business. Our teams are extremely lean and we help our engineers hit the ground running. For example, we have a friendly competition to see how fast new engineers can deploy code to actual customers. The current record is around 48 hours, held by a college hire.

Given Groupon’s relatively young age, we don’t have archaic legacy codebases lying around. This allows us to work with the latest and greatest technologies such as Node, Elasticsearch, and Swift. Don’t get me wrong, there’s plenty of duct tape around when you move this fast, but we work hard to not let it slow us down.

Twice a year the company hosts a week-long hackathon called GEEKon. GEEKon gives our teams the chance to innovate bottoms-up and unleash their creativity and passion for collaboration on new and useful solutions for the company. This year my team shipped Getaways Explorer, a Chrome plugin that displays a beautiful travel deal every time you open a new tab. It was an awesome experience to be part of a multidisciplinary team that went from idea to shipped product so quickly.

How is your role/team different now than it was 3 years ago? 

The biggest difference is the product and engineering teams have grown from a dozen to about 50 employees. The scope and scale of our impact is much bigger. It’s great in that it gives you a better means of competing in the market, but it also brings higher expectations and some growing pains.

We also have a much clearer vision for where we want to take Groupon and Getaways. We want Groupon to be a daily habit for everyone, whether you’re looking for great deals to buy, eat, or experience something. Getaways will simply help you to do this outside your hometown.

How does Groupon’s culture resonate with you? 

Groupon is a quirky place and we all enjoy a good laugh. Whether it’s growing mustaches for Movember or sending mildly inappropriate GIFs in our vacation mails, we don’t take ourselves too seriously. We make sure to hire people who will embrace our culture and help us evolve it.

On the tech side, my favorite cultural aspect is how teams work with each other. For example, if you’re dependent on a team that doesn’t have the bandwidth to build what you need, they’ll often invite your team to write the code yourself and they’ll review the work. That’s rare in companies our size and it allows us to (mostly) avoid dependency hell.

Anything else you want to add?  

Last year Groupon sent me to my 31st country on my 31st birthday. A group of us went to Chennai, India to work with some of our partners out there. It was an incredible experience and as a proud carnivore, I came home with a new appreciation for vegetarian cuisine.

Thanks for reading the People Blog—a blog about people at Groupon. If you’re a fan of the movie “Inception” and want to work for a startup within a startup, check out our current Getaways Tech openings: Senior Product Manager – Consumer Experience, Senior Software Engineer – Front End, Software Development Engineer – Back End.

Tackling Travel in Seattle

Tackling Travel in Seattle

Name: Santosh Ameti
Location: Seattle
Title: Senior Software Development Manager
Start Date: June 2015

What brought you to Groupon from Amazon and Microsoft?

In a short span of 7 years, Groupon has pioneered a new industry and remained the dominant leader in the space. The potential the company has and the growth it has exhibited thus far is exemplary. Local is very critical and important for any neighborhood and in the era of internet taking over physical stores, Groupon is doing a wonderful job  helping local merchants improve their visibility and increase foot traffic. This has created a new way of efficiently spending marketing dollars for any small business shop and in turn having a direct positive impact on their business.

Groupon is a great marketing engine to get foot falls into any local store with minimal cost. The potential the company has is tremendous; what we have done so far is really a drop in the bucket in this space and there is still a ton to achieve. All of these challenges at such a massive scale are very exciting.

What do you work on and what does your team do? 

I manage the Inventory Platform for Getaways. My team focuses on creating a generic platform that can scale across geographies and meet the growing demands of Groupon Getaways. We have various types of travel offerings from diversified sources across the globe; these are consumed by various systems to present to our consumers. It’s a great deal of detail one needs to incorporate to make the same solution work across different countries with minimal or no modifications. We focus on building the next generation platform that enables us to provide our customers with a wide variety of travel deals and bundle it with our amazing local deals.

Can you describe the culture here? 

There are two things to call out: one is the people and the other is the environment. It’s a vibrant workplace filled with enthusiastic people. Groupon has an amazingly fast-paced environment and yet still provides a great work life balance. The environment encourages us to innovate and keep ourselves up to date with the latest and greatest in the technology world. People here are smart, enthusiastic, and collaborate with each other instead of competing; they are approachable and always available to extend their helping hand.

What’s unique about working on Getaways? 

One unique thing about travel is it’s global yet still local to the region. We are placed in a strategic way to extend the local relationships we have with travel and offer a complete solution to our customers. Getaways started about 3 years ago when we realized that our customers were interested in travel deals. Within a short span it became a key growing business and remains a key focus for the company. We are working on extending the solutions we built for the US to global countries and improve the selection for our customers.

What’s challenging about working with travel and what’s exciting about it? 

I believe there are several challenges in the travel space to tackle and we have a chance to create a great solution for our customers. Today you have to visit multiple websites to plan travel, read reviews, reserve activities…it’s a time consuming activity. With our great partnerships we have the potential to be the one-stop place for all travel needs.

Anything else? 

Outside work I like to travel and try different cuisines (I’m vegetarian!) 🙂

I write poems whenever I’m in the mood and have self-published a kindle book. I like playing games indoor and outdoor and am an ardent follower of Cricket.

Thanks for reading the People Blog—a blog about people at Groupon. Interested in working with Santosh on Getaways challenges? He’s actively looking for an Engineer and Engineering Manager, both Backend for Getaways.