Blog Post - Nick Vincent, Jan 9 2018

We launch our Data Engineering division

We launch our Data Engineering division

We are delighted to announce the creation and launch of a new international business offering – Data Engineering - which exploits our unique expertise in this area for the benefit of clients around the globe.

With significant growth opportunities in high-availability data services, the new division will focus on a number of sectors globally including telco, energy, government, large-scale manufacturing, IoT manufacturers and other audiences such as customer services, marketing and IT operations.

The division consisting of 44 team members based across London, Sofia and Seattle, will be working closely with Wunderman in Seattle to extend our Data Engineering service offerings across the US, as well as building a business pipeline in the UK and Europe.

We took a few minutes to ask our Global Head of Data Engineering, Nick Vincent, more about the service...

Can you explain what Data Engineering is?

In the past, the term “data” usually meant a piece of information that a user had intentionally stored, saving documents and finite pieces of information in structured data stores. Even in large systems, this amounts to perhaps hundreds of thousands or possibly a few million entries to store.

Today, data is woven into everything. It’s not just intentionally stored events. It’s part of the fabric of everything we do, every action we take, every place we go, every device we interact with.

Now, even services with only a few million users can easily be generating and making use of billions of pieces of data each day.

Data Engineering is the art and discipline of creating systems that can smartly route, persist and enable low latency operations against large volumes of data. We do this by using strategies that help businesses make good decisions about operational complexity and cost along the way.

Why have we established a Data Engineering division?

We established Data Engineering as a natural evolution of a core team that has proven excellence in creating high-performance applications, infrastructure-as-code, data store implementations and even operational teams that enable businesses to do amazing things with high volume data streams.

Our team has been deeply engaged in transformational change at scale for our customers; not just as a development vendor, but also offering our thought leadership with our customers on things like adopting API-first development strategies, and even auditing potential technical partners. We’re driven by delivering things that work right the first time, at scale.

What services will the division offer?

The services and solutions that this team provides are high transaction volume, high resilience, microservices-based software solutions, aimed at capturing huge volumes of data in real time and making that data accessible to improve and enhance a variety of business-related purposes.

The services are typically cloud-based, to leverage the cost-effective scaling necessary to manage huge real-time data volumes, predominately serving the needs of the customer services organization.

The division will provide:

  • High-level strategic consultancy
  • Architectural consultancy and technical solution design
  • Agile solution development
  • Site reliability engineering

Example practical applications would be for:

  • Capture, storage and processing of time series data
  • IOT data collection
  • Data lake creation
  • Decoupling systems
  • High-performance streaming analytics
  • Highly scalable serverless architectures

Some of the technology stacks we work with include:

  • AWS Lambdas
  • Java (with Spring Boot)
  • Apache Kafka
  • Spark
  • Cassandra
  • HA Proxy
  • Mesos
  • Marathon
  • Consul
  • Zookeeper
  • React
  • ElasticSearch
  • Logstash
  • Kibana
  • Lex (Alexa skills)
  • Lemur and Vault
  • and a long list of other AWS expertise.

What’s driving the need for Data Engineering today?

With the change in volumes of data, and the growing risks of improper security controls around this data, businesses need to be smart about what they collect, how long they keep it, who has access to it, and what controls they have around both their application development security and the operational security of their data platforms.

We work to understand real business needs and align them with what those needs really mean specifically when it comes to:

  • Data latency tolerance
  • Data storage & retention
  • Operations against the data
  • Application & infrastructure security
  • Infrastructure resiliency
  • Operational telemetry

Then we align each one of those needs with the operational budget constraints our customers need to honour.

Businesses are reaching deeper than ever before to use data to open new opportunities and drive material operational improvements in areas that include:

  • Customer care channels
  • Supply chain management
  • Capital improvement decisions
  • Digital content delivery
  • User behaviour analytics
  • Marketing research and analytics

Our team excels at crafting reliable solutions for businesses that want to use data to be better, faster and stronger.

What are the benefits to be gained by businesses investing in Data Engineering?

We believe that there are new opportunities to help organisations capture previously thought-of as uneconomic data streams that can help them provide greater sales, better customer service, or deeper insight of their customers’ interactions with their brand.

Data Engineering done right can help businesses drive positive changes to their bottom line in a number of ways, some of these include:

  • Improved customer experiences in mobile & web apps
  • Faster supply chain lifecycle for returns and exchanges
  • Faster resolution of customer care engagements
  • Faster & better decisions about product development
  • Faster & better decisions about capital improvements
  • Higher ARPU using smarter, faster targeting

find out about our data engineering solutions