How to harness big data in logistics

In an earlier blog on digitalization in logistics, we looked at art: the art of creating added value by making new connections between old, commonplace objects.

This article is all about rock’n roll, and a new attitude to life when data integration, business intelligence, big data and hybrid integration platforms combine to create revolutionary freedom in logistics.

Digital logistics – The Show Must Go On

In recent years, a number of logistics companies have succeeded in integrating customer systems and customer processes. Essentially, this is because it’s customer orientation and price that determine whether an order is placed or not.

As a result, almost all logistics companies now use software and/or cloud services that let them integrate customer requirements. For several years now, the question has not been”if”, but “how”.

Surprisingly, internal system integration – and by extension, of internal data, is lagging far behind. This blog looks at why this is so, and how to change it.

Data diversity – Another One Bites the Dust

Data is an incredibly diverse commodity. Data is of varying quality, is from different internal and external systems, and is sometimes only available temporarily.

This makes it difficult to research even quite basic questions, such as what products are available at what quantities at what location, or even the KPIs of individual locations.
Or, there is a lack of (real-time) knowledge about where goods and objects are located. Can you and your customers access this data yourself 24/7? Or, do you need to phone the right person in the right department during office hours and do other spot research?
Do you know where each of your service technicians are located and what specific skills they have? This would let you send out exactly the right person for a malfunction.
…

We see data management in digital logistics as making use-case relevant company data available for internal and external stakeholders, potentially in real time.

Data management and big data – A Kind of Magic

The first step towards answering the seemingly simple questions above is the ability to merge different data streams. Without this step, more advanced issues such as data management, advanced data analytics (ADA) or big data simply aren’t possible.

While traditional data analysis tends to involve evaluating past events, Advanced Data Analytics (ADA) includes data mining, machine learning and statistical methods for predictive and prescriptive evaluation. This lets you use your big data for planning and forecasting.¹ However you approach these topics…

…It all begins with data

Technical and organizational success factors for Advanced Data Analytics

Figure: A BVL survey of success factors in ensuring relevance and implementation of data management²

As the matrix above shows, the orange dots for data matters such as quality, availability, variety, security, amount and technical infrastructure

have the highest relevance (graded with 4 to 4.5 points on a scale of 0-5)
have, at 40%-50%, the highest implementation rate,

If we look at where the blue organizational dots with the highest relevance appear on the implementation scale, namely ‘integration into existing processes‘ and ‘top management support‘, we can see where there’s still work to be done.

Data integration – Radio Ga Ga

On the one hand, according to the BVL survey³ above, a good one third of companies are already integrating data management into existing processes. On the other hand, this also means that the majority (almost two-thirds) are not yet doing so. What is making this task so difficult for the logistics sector?

Basically, the task is to ensure data availability, security, quality, and diversity for all systems and processes.

What makes it tricky is the sheer wealth of systems to be connected in a typical company, and the many ways this could be done

for internal systems including

warehouse management system (WMS)
transport management system (TMS)
order picking system
Enterprise Resource Planning system (ERP)
pick by system (Pick-By)
customer relationship management database (CRM)

and external systems such as

customers‘, suppliers‘ and partners‘ systems
various business processes
various, often industry-specific formats for data exchange
on premises or cloud-based-systems

using different integration technologies

If we look at this mathematically, the number of point-to-point connections needed would increase quadratically to the number of systems we want to integrate!

A good integration platform reduces this quadratic growth to simpler linear growth, the best possible outcome.

The question we are left with is then: could your integration services or platform cope with integrating internal services from data management to data lakes to enable ADA, big data and the rest?

Hybrid Data Integration Platforms and Services – Bohemian Rhapsody

The quality of your integration platform and services is crucial to the success of your digitalization strategy. The following questions help you future-proof your strategy:

Can you simply and seamlessly integrate all the backend systems listed above?
Can you move and provide data between internal applications, cloud services and data sources in real time?
Can you electronically create and exchange business documents to industry standards?
How securely is confidential data exchanged between systems and/or people?
Are you able to integrate devices, equipment and objects into a process to create an IoT-based logistics 4.0?

If you have answered all these questions with ‛yes’, the congratulations! You’re ahead of most of the competition.

You already have a scalable, modular platform or service partner for all your business process digitalization you are free, not beholden to a specific interface, data format or communication protocol.
Why don’t you celebrate with Queen’s back catalogue?

If you haven’t answered ‛yes’ to all questions – don’t worry, there are many other companies also under pressure, wanting to break free.

From an organizational perspective, you will need to have senior management on your side to implement a data management strategy. However, because of the general, widespread rise in digitalization, this is probably easier now than just a few years ago.

From a technical perspective, take a look at the wealth of information in