Think big, start small, scale fast:10 steps to a scalable, sustainable data architecture

I consistently hear that many manufacturers are drowning in data and struggling to make it useful. Why is that?

A modern industrial facility can easily produce more than a terabyte of data each day. With a wave of new technologies for artificial intelligence and machine learning coupled with real-time dashboards, industrial companies should be seeing huge gains in productivity. Unplanned asset and production line maintenance should be a thing of the past. But that is not the case.

Access to data does not make it useful. Industrial data is raw and must be made fit for purpose to extract its true value. Furthermore, the tools used to make the data fit for purpose must operate at the scale of an industrial enterprise. For many industrial companies, this is a daunting task. Based on the best practices we have learned from our customers over the last several years, here are 10 steps to achieving a scalable, sustainable data architecture.

As with any major initiative, ensuring that your project is aligned with corporate goals must be the first step. Make sure the right cross-functional stakeholders are in the room from the beginning, and that all stakeholders agree to prioritize the project and can reach consensus on the project goals.

Lack of architectural strategy is one of the most common sources of failure for Industry 4.0 initiatives. Taking the time to plan your architecture before you start your project pays dividends as you expand to other areas and sites. A well-planned data architecture will deliver the visibility and accessibility necessary to give your people and systems the data they need when they need it.

Your use case may be driven by assets, processes, or products. Regardless, company stakeholders should identify the project scope, applicable data that will be required, and the target persona who will consume this information and act on it. Use cases range from predictive asset maintenance to first-pass yield—to emissions reductions and other sustainability initiatives. This first use case is how you will prove value and return on investment.

This approach is contrary to traditional data acquisition approaches that would have you begin with source systems. Focusing on your target systems first will allow you to identify exactly what data you need to send, how it must be sent, and at what frequency it should be sent, so you can determine which sources and structure are best suited to deliver that data. Focusing on the target system and persona will also help identify the context the data may require and the frequency of the data updates.

You can better understand the specific challenges you will need to overcome for your project by documenting your data sources. These challenges may include data volume, correlation, context, and standardization. Document data sources and their nuances now to prevent future obstacles to success.

This is the step where a DataOps solution should be evaluated. A DataOps solution acts as an abstraction layer that uses APIs to connect to other applications while providing a management, documentation, and governance tool that connects data sources to all required applications. DataOps solutions should be purpose-built to move high volumes of data at high speeds with transformations being performed in real time while the data is in motion.

Once your project plan is in place, you can begin system integration in earnest by establishing secure connections to the source and target systems. It is vital that you wholly understand the protocols you will be working with and the security risks and benefits that come with them. Security is not just about usernames, passwords, encryption, and authentication. It is also about protocol selection and integration architecture.

The first step in modeling data is to define the standard data set required in the target system to meet the project’s business goals. The real-time data coming off the machinery and automation equipment is typically at the core of the model. These models also should include attributes for any descriptive data, which are not typically stored in the industrial devices but are especially useful when matching and evaluating data in the target systems. Once the standard models are created, they should be instantiated for each asset, process, and/or product relevant to the use case. This task can be accelerated with a DataOps solution.

When the instances are complete, data flows control the timing of when the values for an instance are sourced, standardized, contextualized, calculated, and sent to the target system. Data payloads may be published cyclically, when an event occurs, or only when a change occurs. A well-built data flow will retain the semantics of a model while transforming the presentation and delivery to the unique needs of the systems consuming it.

Now that your use case is officially off the ground, it is vital that you pay close attention to your results and track your progress.

There are many definitions and meanings for “sustainability.” As my colleague John Harrington has pointed out, sustainability in manufacturing is not just about cutting emissions. By reducing defects and scrap through standard Lean and Six Sigma approaches, companies can also support their sustainability goals by expending less power per unit while increasing production. These projects will enable customers to make progress on their sustainability initiatives—and improve their bottom line.

At HighByte, we believe a sustainable architecture is the foundation for industrial sustainability, helping industrial companies operate more efficiently and reduce both physical and digital waste. So, let’s take the first step and get started together.