How to build the right big data architecture

Architecture. Infrastructure. Ecosystems.

The words we use to describe technology often come from other parts of life. But, they can be very helpful in understanding the role technology and big data play in enabling business operations and improving performance.

In the case of a big data architecture, the essential idea is that documenting the right foundation of architecture, infrastructure and applications ultimately allows the business to more effectively use big data more on an everyday basis. While big data architecture can seem like a highly technical topic, it’s important to realize that big data innovations and insights are not possible without a well-conceived, clearly defined and thoughtfully designed architecture. So, if your business has big plans for big data, a strong big data architecture is required to executing those plans.

What is Big Data Architecture? And Why Do I Need It?

As with many big data and technology-related terms, it’s worth clarifying the meaning of big data architecture. Like the blueprints for a house or building, a big data architecture is a conceptual or graphical model of how big data and other information assets will be captured, stored, managed and made accessible to various user groups and applications. Typically, big data architectures outline the hardware and software components that are necessary to a full big data solution. Big data architecture documents may also describe protocols for data sharing, application integrations and information security.

If that sounds a tad dull, it is worth remembering that no one would build a house without blueprints. As such, no one should plan to effectively leverage big data without a big data architecture. And the more you’re investing in a house (or big data solutions for that matter), the more you need a big data architecture to make sure you get the ROI you desire. In other words, big data architectures help ensure data flows as planned so the right users can access it via the right tools.

How to Get the Right Big Data Architecture: Ask the Right Questions?

Designing the big data architecture you need often begins with asking the right questions:

• How do big data strategies – the business problems to be solved, operations to be improved and objectives to be achieved by using big data – shape architecture needs?
• Which of our existing data sources and systems can be “plugged into” an integrated architecture for big data?
• How do we account for new data sets – like sensor data or data from the Internet of Things?
• How can our approach to big data architecture help move data-driven and analytics-enabled thinking into the center of our business?
• What are the required components to “operationalize” or scale big data and analytics program beyond pilot phases?

One proven approach is to think in terms of a unified data architecture (UDA) that can generate more actionable insights from big data. A unified data architecture is built around essential components, like an integrated data warehouse and a platform for analytics and discovery, which collectively bridge the gap between raw data sources and specific business intelligence tools and standard CRM applications. This dramatically reduces the complexity of traditional “hybrid” environments and enables companies to ingest extremely fast-moving datasets and offer users cross-platform access to data and analytics engines.

Big Data Architecture in Action

As important as big data architectures are, business and IT must not mistake the blueprint for the solution itself. A big data architecture is the design and documentation describing how big data works once all the components, data sources and applications are connected and integrated in a unified whole. The architecture doesn't necessarily generate business value on its own, but it sets up a foundation for success. The value comes from defining the right big data processes and structure, deploying advanced big data analytics and having the right people and teams in place to interact with and interrogate the data. That’s how users can solve problems, find new opportunities, make better decisions and take other meaningful decisions.

Consider how large retailers need robust and flexible big data architectures to better understand consumer behavior across devices and channels (including in-store). Personalizing marketing campaigns or offering coupons in real time require integrated data and sophisticated targeting which aren’t possible without an advanced big data architecture.

The world's largest telecom, Verizon uses its big data architecture to “listen” to 100 million customers. The logical model that ensures both structured unstructured data is stored in the right place and it’s accessible and can be explored via discovery platforms.

Netflix Designs a Big Data Architecture to Drive Engagement:

Netflix’s big data architecture features a cost-effective collection of services, platforms, applications, and tools for smarter data management, processing, and analytics. A unique analytics platform is requited because analysts use a range of approaches to solve various problems, even as the overall environment handles a staggering workload.

A Big Data Architecture Means Building a Foundation for the Future

It is important to recognize that big data architectures are long-term commitments, not “one-and-done” projects. Big-picture and long-term thinking confirms that big data architectures should be comprehensive and capable of solving many business problems – including those that will arise in the future. In other words, the big data architecture of today must be designed to take advantage of new tools and technology in the future and incorporate new varieties and expanding volumes of data. Your big data architecture is one way to prepare for your journey into the future of big data.

Curious about how Teradata can help you?

Learn about our Big Data Solutions