Job Description

Job Title: Senior Data Architect

Location: London

About the Job you are considering:

We are seeking a Senior Solution Architect with deep experience designing and delivering enterprise-scale big data, streaming, and real-time data integration solutions. The role is focused on solution architecture across Cloudera-based big data platforms, Kafka-based streaming applications, and real-time data integration / analytics solutions, including cloud-hosted data platforms such as Cloudera on AWS or equivalent AWS-native services. This is not a pure data modelling, reporting, analytics, or governance role. The successful candidate must have a strong architecture background and be comfortable operating across both solution design and hands-on technical delivery.

Hybrid working:

The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.

Key Responsibilities

• Own the end-to-end solution architecture for large-scale distributed data, streaming, and data integration solutions.
• Design solutions across Cloudera CDH/CDP, data lakes, operational data stores, event streaming platforms, and real-time data pipelines.
• Architect and document solutions using technologies such as Hive, Impala, HBase, Spark, NiFi, Flink, Kafka, and equivalent AWS services.
• Design Kafka-based streaming applications, including event ingestion, topic design, consumer patterns, error handling, replay, scalability, and resilience.
• Define solution patterns for Spring Kafka consumers, ingestion into HBase, and consumption through Spring Boot / API-based services.
• Provide architecture leadership across data ingestion, streaming integration, transformation, storage, APIs, and downstream consumption.
• Work with engineering teams to ensure solutions are deliverable, scalable, secure, supportable, and aligned to enterprise architecture principles.
• Review technical designs produced by engineering teams and ensure alignment with the approved solution architecture.
• Identify technical risks, dependencies, assumptions, non-functional requirements, and design trade-offs.
• Support delivery planning by helping break solutions into iterative delivery increments and sprint-level outcomes.
• Engage with business stakeholders, programme teams, enterprise architects, engineering leads, and governance forums.
• Build, represent and own solution designs (Architecture Summaries and High-level designs) to architecture review boards and senior technical stakeholders where required.
• Develop and promote architectural patterns and best practices and engineering standards across big data, streaming, integration, APIs, and cloud-based data platforms.

Essential Experience

Candidates must demonstrate the following

10+ years’ overall IT experience, with significant experience in data platform, integration, or distributed systems delivery.

• 5+ years’ experience in solution architecture, technical architecture, or lead design roles.
• 2+ years’ experience architecting streaming or real-time data applications, preferably using Kafka or equivalent event streaming technologies.
• Proven experience designing solutions on Cloudera CDH/CDP or equivalent big data platforms.
• Strong demonstrable experience with some of the distributed data technologies (if not all) such as:
• HBase
• Spark
• NiFi
• Kafka
• Flink
• Hive
• Impala

• Experience designing and delivering large-scale data ingestion, transformation, storage, and consumption patterns.
• Experience with Kafka consumer architecture, including consumer groups, offsets, retries, dead-letter handling, ordering, partitioning, and scalability.
• Experience with API-based data consumption, ideally using Spring Boot / Java-based services.
• Strong understanding of batch, micro-batch, and real-time streaming architectures.
• Experience working in complex enterprise environments with multiple systems, vendors, platforms, and governance processes.
• Strong understanding of non-functional requirements including performance, scalability, resilience, security, observability, and operational support.

Required Technical Skills

The successful candidate should have hands-on exposure to several of the following:

• Cloudera CDH/CDP, ideally on AWS or hybrid cloud environments
• Apache Kafka or AWS equivalents such as MSK, Kinesis, EventBridge, or similar
• Spark / Spark Streaming
• HBase / Cassandra / NoSQL data stores
• Hive / Impala
• Apache NiFi / ingestion frameworks
• Apache Airflow Orchestrations
• Apache Flink or equivalent stream processing technologies
• Java / Spring Boot / Spring Kafka
• Python and PySpark
• REST APIs and service-based data consumption
• Change Data Capture and event-driven integration patterns
• Data lake, ODS, and distributed storage design
• Cloud data platform design, preferably on AWS

Desirable Experience

• Financial services or banking experience.
• Experience with regulated, secure, enterprise-scale data platforms.
• Experience presenting to architecture governance boards.
• Exposure to enterprise data governance, lineage, metadata, and data quality tooling.
• Experience with DevOps, CI/CD, automated testing, and modern engineering delivery practices.
• Experience with cloud-native data services on AWS.

Candidate Profile

This role is suited to a hands-on Senior Solution Architect who can design and guide the delivery of complex data and streaming solutions.

The ideal candidate will:

• Be an architect first, with strong hands-on engineering awareness.
• Have designed real production solutions using Kafka, Cloudera, Spark, HBase, APIs, or equivalent technologies.
• Understand both architecture-level trade-offs and engineering-level implementation constraints.
• Be comfortable challenging, reviewing, and improving technical designs.
• Be able to work with senior stakeholders while also engaging deeply with engineering teams.
• Have practical experience delivering scalable, resilient, secure, real-time or near-real-time data platforms.

Important Clarification

This is not a role for candidates whose experience is primarily in:

• Data analysis
• Data modelling only
• BI reporting
• Dashboard development
• SQL reporting
• Data governance only
• Business analysis
• Power BI / Tableau reporting
• Conceptual data architecture without implementation experience

Candidates must have demonstrable experience in solution architecture for distributed big data platforms, streaming systems, event-driven integration, or real-time data applications.

We are a Disability Confident Employer

Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Government’s Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:

• Declare they have a disability, and
• Meet the minimum essential criteria for the role.

Please opt in during the application process.

Make It Real (what does it mean for you):

• You’d be joining an accredited Great Place to work for Wellbeing in 2024. Employee wellbeing is vitally important to us as an organisation. We see a healthy and happy workforce a critical component for us to achieve our organisational ambitions.
  To help support wellbeing we have trained ‘Mental Health Champions’ across each of our business areas, and we have invested in wellbeing apps such as Thrive and Peppy.
• You will be empowered to explore, innovate, and progress. You will benefit from Capgemini’s ‘learning for life’ mindset, meaning you will have countless training and development opportunities from thinktanks to hackathons, and access to 250,000 courses with numerous external certifications from AWS, Microsoft, Harvard Manage Mentor, Cybersecurity qualifications and much more.
• You will be joining one of the World’s Most Ethical Companies®, as recognised by Ethisphere® for 13 consecutive years. We live our values by making ethical business choices every day. Working ethically is at the centre of our culture at Capgemini, meaning you will be helping to create a future we can all be proud of.

Why you should consider Capgemini:

Growing clients’ businesses while building a more sustainable, more inclusive future is a tough ask. When you join Capgemini, you’ll join a thriving company and become part of a collective of free-thinkers, entrepreneurs and industry experts. We find new ways technology can help us reimagine what’s possible. It’s why, together, we seek out opportunities that will transform the world’s leading businesses, and it’s how you’ll gain the experiences and connections you need to shape your future. By learning from each other every day, sharing knowledge, and always pushing yourself to do better, you’ll build the skills you want. You’ll use your skills to help our clients leverage technology to innovate and grow their business. So, it might not always be easy, but making the world a better place rarely is.

About Capgemini:

Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organisations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,000 team members in more than 50 countries. We deliver end-to-end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations. The Group reported 2024 global revenues of €22.1 billion.
Make it real | www.capgemini.com