In September last year, Confluent, a leading provider of data streaming solutions, introduced ‘Data Streaming for AI’, a new initiative to speed up organisations’ creation of real-time AI applications. More recently, the company announced the general availability of Confluent Cloud for Apache Flink. This fully-managed service enables customers to process data in real-time and create high-quality, reusable data streams.
Behind all the innovations in this space is Confluent’s strong and resilient AI and analytics team. “Building a truly data-driven culture is one of the top priorities for Confluent’s Data team. A critical part of achieving that is applying data science to address real-world requirements in business operations,” Ravi Kiran Yanamandra, manager of data science for product and growth marketing at Confluent, told AIM.
Yanamandra, along with Karthik Nair, director of international talent acquisition at Confluent, took us through the company’s AI applications, hiring process, skills needed, and work culture.
The company is seeking data scientists and engineers to further bolster its tech team.
Inside Confluent’s Data Science Wing
The data team at Confluent is structured into sub-teams specialising in data engineering, data science, and business intelligence.
The organisation has leveraged data science in experimentation to inform product decisions and optimise marketing investments across multiple channels. This involves a multi-channel attribution model and an improved predictability model built on key business KPIs through machine learning forecasting models, enabling more precise planning.
Additionally, machine learning forecasting models improve the predictability of critical business KPIs.
In terms of implementation, Confluent’s data science team uses a combination of online and offline machine learning models to support various aspects of the business. For example, online algorithms are deployed to evaluate the quality of leads or new signups in real time, allowing for immediate actions based on the insights generated.
Furthermore, offline models are operationalised to assist business partners in making informed decisions, such as guiding marketing spend decisions through the marketing channel attribution model and providing predictive insights into future performance through revenue forecasting models.
“While still in the experimental phase, we are actively exploring the potential of generative AI as a productivity tool,” said Yanamandra, highlighting that the initial applications include enhancing internal search capabilities and evaluating the quality of support provided through channels like chatbots and email communications.
Moreover, through its comprehensive data streaming platform, organisations can stream, connect, process, and manage data in real time, creating innovative solutions previously considered unattainable. By integrating generative AI with data streaming, organisations can swiftly address inquiries with up-to-date and comprehensive information.
In addition to leveraging existing technologies, the team also builds proprietary models using its proprietary data assets to address specific business challenges, said Yanamandra. These models, such as consumption forecasting and lead scoring, are tailored to Confluent’s unique needs, further enhancing their competitive advantage in the market.
The team predominantly uses SQL and Python for modelling and analysis, supported by tools like dbt, Docker, and VertexAI for data pipeline management and production model deployment. Tableau is the primary platform for visualisation and reporting, enabling stakeholders to gain actionable insights from the data effectively.
Interview Process
“The hiring process for our data team focuses on assessing candidates in three areas – technical, analytical, and soft skills,” commented Yanamandra.
Candidates are evaluated based on their proficiency in Python and SQL, experience in ML algorithms and data modelling, and familiarity with A/B testing and data visualisation. Analytical skills are assessed through problem-solving abilities and structured thinking, while soft skills, such as business understanding and communication, are also crucial.
The interview process begins with technical screening focussed on SQL and Python, followed by a real-world business scenario assessment. For data science roles, there’s an additional stage dedicated to statistics knowledge and machine learning abilities. The final interview with the hiring manager evaluates project delivery experience, technical leadership, motivations, and cultural fit.
Expectations
When joining Confluent’s data science team, new members can expect to actively engage with business partners, focusing on solving their specific business challenges. Successful candidates join as subject matter experts for the company’s data tools and technologies, and training is provided to deepen their understanding of the relevant business domain.
New joiners can expect to work in a “highly collaborative, innovation-driven, and fast-paced environment on the data science team. We move quickly and prioritise translating data insights into tangible business impact”, Yanamandra added.
“Another unique aspect is that candidates are exposed to diverse domains, offering opportunities to collaborate across functions such as marketing, sales, finance, and product analytics,” Nair told AIM.
Mistakes to Avoid
While interviewing candidates, Yanamandra has noticed a common pattern. Candidates often assume that proficiency in technical skills such as SQL, Python, or machine learning is the sole criterion evaluated for data science roles.
However, while these skills are definitely crucial, Confluent equally prioritises problem-solving abilities and the capacity to apply data science concepts to practical business scenarios.
Work Culture
Confluent strives to prioritise hiring individuals who demonstrate empathy and interact well with others, fostering a collaborative and inclusive environment. “As a rapidly growing company, our employees are self-motivated and driven to seize market opportunities. We follow unified decision-making and communication with an open and hierarchical-free structure,” Nair told AIM.
Nair stated that the company also offers flexibility through a ‘remote-first’ policy, allowing employees to work from various locations. Alongside competitive benefits, it ensures each employee’s contributions are recognised and valued.
“Our team thrives on a culture of intellectual curiosity and innovation, where individuals will be encouraged to push the boundaries of what’s possible,” said Nair. The company strives to build an equal, diverse and inclusive work culture.
“We’re a high-growth company with incredible results and achievements, yet only scratching the surface of our potential impact in a rapidly growing market. Joining us promises an exhilarating journey of growth,” concluded Nair.
If you think you are a right fit for Confluent, apply here.
