Monday, March 24, 2014

NASCAR Attains Intimacy and Affinity with Fans Worldwide Using HP HAVEn Big Data Analytics

Transcript of a sponsored BriefingsDirect podcast on how NASCAR is using big data and analytics to learn from and engage with their vast fan base.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Discover Podcast Series. I’m Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing sponsored discussion on IT innovation and how it’s making an impact on people’s lives.

Gardner
Once again, we’re focusing on how companies are adapting to the new style of IT to improve IT performance and deliver better user experiences -- and business results. This time, we’re coming to you directly from the recent HP Discover 2013 Conference in Barcelona.

We were in Barcelona the week of Dec. 9 to learn directly from IT and business leaders alike how big data, mobile, and cloud -- along with converged infrastructure -- are coming together to help support their business goals.

Our next innovation case study highlights how auto racing powerhouse NASCAR is learning more about its many fans, and is better able to tailor its services and keep connected to that fan base using big-data analysis. To learn more about what they're doing and how they're doing it, please join me in welcoming our guest, Steve Worling, Senior Director of IT at NASCAR, based in Daytona Beach, Fla. Welcome, Steve.

Steve Worling: Thank you, Dana. I appreciate the invite, and I’m glad to be here today to really tell our story about what we're doing with big data.

Gardner: Let's start with the big picture then. NASCAR has been around for quite a while. In your business, like many, connection to your customers has always been desired, but nowadays we seem to be able to do it in entirely ways. That can be good thing, or that can be a bad thing. Tell us about the context of what you're trying to do with your fan base, and then how technology comes to bear on that.

Worling: NASCAR has been around for 65 years, and we have probably one of the most loyal fan bases out there. NASCAR really wants to understand what our fan base is saying about our sport. How do we engage with them, how are we really bringing our sport to their entertainment, and what's the value of that?

Worling
So NASCAR partnered with HP to build a first-of-its kind of Fan and Media Engagement Center. That’s a new platform for us that allow us to listen to the social media outlets -- Twitter, Facebook, Instagram, all of those social media outlets -- to understand what the fans are talking about.

Something unique about this platform is that it also allows us to bring in the traditional media news sites. What is the media saying about our sport, and then how do you tie those conversations together to get a really nice single pane of glass on the overall conversation? What are our fans are saying, what are the news media saying, and how does that help and benefit our industry as a whole?

Gardner: What's the scale here? Obviously, you've talked about using any social media you can get. It sounds like you don’t want get some of the data -- but all of the data. How big is that?

Want to know everything

Worling: We absolutely want to know everything that’s being said across all of those platforms. We saw about 18 million impressions in our first year of the platform. That’s impressions across the social side and the news-media side. It was big, and this was our first year at it.

On the news media side, we're only collecting from a small sample right now. Next year, we're going to really enhance that and grow that from a few different news sites to hundreds of sites, as well as start to bring a more of awareness to our fans around social interaction.

So we're expecting to see that number grow significantly. This year, as I said, a solid 18 million tweets overall translates to about 110,000 tweets during a race day, even up to about 15,000 tweets per minute.

Gardner: Just to be clear, this is a global audience, and I believe you're trying to connect and expand into more areas of the globe.

Worling: NASCAR is a predominantly US-based sport, but we are growing internationally. Today, we have a series in Mexico. We have a series in Canada as well, and we just expanded into Europe with our Whelen Euro Series.

This platform will also help us engage and understand how the sport is performing in those markets. What's the sentiment of the fans? It's really a great platform to allow us to right anything that we might be doing wrong. So if we need to enhance the marketing or enhance the engagement of those tracks, we're able to do that through this platform.
If we need to enhance the marketing or enhance the engagement of those tracks, we're able to do that through this platform.

Gardner: I've talked to so many companies that thought they knew their customers, but didn’t. When the data was available to them, they learned new  things. But then, even more so, when you have a cultural divide, you couldn't even anticipate there was an "unknown unknown" element to it. So the data is the only way to really get inferences when you start to go so wide and deep.

Worling: Our sport is unique, because there is a vast community that makes up our sport. You have a NASCAR governing body and that's what I represent. Then, there is a large race track ownership. We call those promoters, and those are the folks who are selling tickets and getting you out to the race track.

Then, we have our teams and our drivers, and those are independent contractors. So you have those that are involved in the sport, and then our sponsors and our partners that help bring all of that together and make this ecosystem. That is NASCAR.

We're able to collect data on all of those different constituents, and then share that value. I’ll give you a great example. This year, HP became a great partner with us around our Fan and Media Engagement Center.

Share the value

Our goal over the next couple years, as we work with HP, is to be able to sit down with them and share the value and what their sponsorship and their partnership brings to NASCAR. We want to develop and grow the relationship for a longer period of term. We give them real data on their activation and involvement in the NASCAR industry.

Gardner: No guesswork is good work. Tell us how you faced this issue of how to do this best. We know how important it is to our business. We know that customer information is being shared in whole new ways. How do you then take the technology and get a handle on it so that you can perform what you want?

Worling: We partner with HP, as I said, to build this platform. We’re leveraging products like their IDOL engine. The Explore capability from their Autonomy platform allows us to ingest all of this different data, put it together, and then really start building that single pane of glass to understand what these conversations are -- whether there is a breaking story around activation within our sport, or something else.

As it's collecting this data, the platform starts to stitch it together so that we can understand what the conversation is. So it’s taking that news outlet information, taking the social sentiment, and putting it together to make sense of it. It’s taking all of that unstructured data, structuring it, and then giving us the analytics that allow us to understand the conversation -- and react appropriately.
One of the new things that I'm excited about is in telling our story is that we've got a great command center.

It could be a story that makes sense and is telling the right story, or it could be a story that needs a little bit of direction from NASCAR to make sure that we're getting the right story out there.

So HP building that with Autonomy has been very valuable. We're getting ready to deploy HP Vertica on top of that now to allow us to take this large amount of data we’re getting and putting it into the Vertica data infrastructure. Then we can start making even more connection points and more rationalization, and then being able to layer other tools on top of it -- things like Tableau Software -- to help us with visualization.

One of the new things that I'm excited about is in telling our story about our great command center. It’s a showcase piece that you can come and see what we’re actually reporting on the analytics. We’re going to build a map of the U.S. that allows us to give us the hotspots of information.

So as people are tweeting, maybe good or bad, in California, you might get a big red spot. We can drive down into that, understand what that data is, and then engage through our dot-com platforms and other media outlets to make sure that we're saying the right story or addressing the concerns that are out there.

Gardner: As a quick aside, we just saw on the stage today here that Facebook put up a very impressive map that was built using Vertica. It shows their actual installed base and the connections between them. Of course, it looks very much like a map of the world, but it's a map of Facebook.

Amazing visualization

Worling: That was an amazing visualization, and I can't wait to be able to do the same thing. I thought that was a really neat and I’d love to be able to get the resolution of the world like they have, but I will be happy to get a great, rich US look. That was totally a cool thing, and I hope that we can do the same thing as well.

Gardner: So one of the great things about what you have been doing is getting all the data. One of the bad things you've been doing is getting all the data. How do you move beyond this being a fire hose and make it actionable? You have Tableau and visualization, but is there anything more?

Worling: As I mentioned, we’re storing everything in IDOL today. We'll be migrating to Vertica shortly to help us with the consumption. For us, this year, it's been a little bit of we just didn’t know what we didn’t know. We weren't really sure what kind of data we were going to see and how we were going to react to it. Our sport is a great sport, but like any sport or any business, there's always a little controversy with it, and we experienced some of that this year. So it was more of a great platform to help us do crisis management.

As we dealt with the situations that came up, we were able to get data from this and react to it appropriately. But we've also started to learn some proactive things to think about.
With Vertica and IDOL, we’re positioning ourselves or have the right platform that allow us to grow extensively as we look to the future.

As we launch a new car this year, our Gen-6 Car, what is the engagement or sentiment from our fans? We’ve been able to do some deep analytic research on what that is and get valuable information to be able to hand GM, who launched this car with us this year and say, “This is the results of the news" instantly -- a lot of big data.

As I said, we have 18 million impressions this year, which was phenomenal, and I don’t think we had a bar to set. Now, we’ve have set the bar for next year and I think with Vertica and IDOL [part of HP HAVEn], we’re positioning ourselves or have the right platform that allow us to grow extensively as we look to the future.

Gardner: I’ve heard from other folks, Steve, that it’s a slippery slope. Once you start getting big-data capabilities and driving more data into it, you get hungry for more data. You’ll start thinking about places to acquire it, doing joins, and then finding even better analysis. Any thoughts as to where you might go next, now that you’ve tapped the social-media environment?

Worling: There are two ways to answer that. One, we’re going to continue to grow the social media side. I mentioned the things that we’re doing today with Facebook and Twitter. Instagram really is the next big piece of integration for us.

For NASCAR, it’s important for us to engage younger people in that Gen Y, Millennial Generations. Instagram is a key component to do that. So that’s going to be a big focus for us in getting that integrated and then just keeping an eye out for the new social solutions or offerings that are coming out and how we keep them integrated.

Traditional media

Then, we’re going to start working on the traditional news media as well. As I mentioned, it’s going to be key for us to understand the press impacts. That’s very relevant for our CEO and Chairman. I didn’t mention, but we’ll also be bringing in video from our broadcast partners. We broadcast nationally in the US, as well as in 198 countries worldwide. That story is very important to us.

We’ll be growing a lot of that next year. The second side of that is our business becomes more aware of this tool. We’ve been getting just inundated with requests, some from the sales guys, as they’re trying to develop new sales, how we should value what it means to be part of our sport. There are renewals in the sales process as well, the value of the partners that are already existing and then taking it to our drivers.

A great story I love to tell is about a young and upcoming driver that started in our Camping World Truck Series. This year has to build his brand. He has a brand that he needs to develop and get out there.
The things that we’re doing today with Facebook and Twitter. Instagram really is the next big piece of integration for us.

We brought him into the Fan and Engagement Center and spent about three or four hours taking him through different analytics, different use cases of information around his brand, and helped him understand what it meant to be good. We showed him the things he needs to develop, and the things that he wasn’t so good at, so he could take that away and work better on those. We’re definitely seeing a lot of requests from the industry: How does this platform benefit them and how do they get rich data out of it?

Gardner: Well, it really seems like a really powerful capability that you’ve only begun to scratch the surface. I wish you well with that.

Worling: Thank you.

Gardner: We have been learning about how NASCAR has been getting into big data and using several HP technologies, including Autonomy IDOL Engine and, increasingly, Vertica, and then visualizing these findings to better improve how it relates to its vast fan base.

With that, please join me in thanking our guest, Steve Worling, Senior Director of IT at NASCAR. Thanks so much, Steve.

Worling: Thank you.

Gardner: And thank you also to our audience for joining us for this special new style of IT discussion coming to you directly from the HP Discover 2013 Conference in Barcelona.

I’m Dana Gardner; Principal Analyst at Interarbor Solutions, your host for this ongoing series of HP sponsored discussions. Thanks again for listening, and come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a sponsored BriefingsDirect podcast on how NASCAR is using big data and analytics to learn from and engage with their vast fan base. Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in:

Monday, March 10, 2014

HP Updates HAVEn Big Data Portfolio as Businesses Seek Transformation from More Data and Better Analysis

Transcript of a BriefingsDirect podcast on how HP is developing products and platforms to help businesses deal with the demands of big data in a competitive environment.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hello, and welcome to the next edition of the HP Big Data Podcast Series. I’m Dana Gardner, Principal Analyst at Interarbor Solutions, your moderator for this ongoing sponsored discussion on how data is analyzed and used to advance the way you live and work.

Gardner
Once again, we're showcasing thought-leaders and companies worldwide that are capturing myriad knowledge, gaining ever deeper analysis, and rapidly and securely making those insights available to more people on their own terms.

Our next big-data innovation discussion highlights how the latest version of HP HAVEn produces new business analytics value and strategic returns. So please now join me in welcoming Girish Mundada, Chief Technology Officer for HP HAVEn.

Girish Mundada: Thanks, Dana. Good to be here with you.

Gardner: And Dan Wood, Worldwide Solution Marketing Lead for Big Data at HP Software. Welcome, Dan.

Dan Wood: Hello, Dana,

Gardner: Dan, let me start with you. We’re in a fascinating time because analytics and big data are now top of mind. What was once relegated to a fairly small group of data scientists and analysts as reporting tools -- and I am thinking about business intelligence (BI) -- has really now become a comprehensive capability that’s proving essential to nearly any business strategy.

From your perspective, what’s behind this eagerness to gain big-data capabilities and exploit analytics so broadly?

Wood: You’re right, Dana, and it’s because we're starting to see some very clear quantification of the value and the benefits of big data. It’s fair to say that big data is probably the hottest topic in the industry.

Wood
There’s a lot of talk across all forms of media about big data right now, but what’s happened is that credible publications like the "Harvard Business Review," for example, have started to put solid numbers around the benefits that enterprises can get if they can get their hands around big-data analytics and apply it to business challenges.

For example, Harvard Business Review is saying that, on average, data-driven organizations will be five percent more productive and six percent more profitable than their competitors.

Worth chasing after

Think about that. A six-percent distinct profitability increase would double the stock price for a lot of organizations. So there really is a prize worth chasing after.

What we’re seeing, Dana, is much more widespread interest across the organization and not just within IT. We’re seeing line-of-business leaders understanding and, in many organizations, actually starting to benefit from big-data analytics.

They’re able to analyze the call logs in a call center, better understand the clickstreams on a website, and better understand how customers are using products. All of these are ways of analyzing large amounts of data and directly tying it to specific line-of-business problems.

That’s where we are right now. Industries around the world are going through transformational projects using big data to gain competitive advantage.

Gardner: It’s interesting too, Dan, that they’re not just taking these as individual data sets and handling them individually, but increasingly businesses are combining them, and finding new relationships, and doing things that they really couldn't have done before.

Wood: Absolutely. It’s the idea of 360-degree view of their internal operations, or of their external customer trends and needs -- and it’s come from combining data sets.
This industry label of big data is perhaps not the most helpful, because it’s not just the volume of data that is the challenge and the opportunity for the business.

For example, they’re combining social media analytics on customers with the call logs into the call center, with internal systems of record around the customer relationship management (CRM) and ongoing customer transactions. It’s by combining all those insights that the real big-data opportunity reveals itself.

Gardner: And the sources for those insights and data, of course, are across almost any type of information asset. It’s not a just structured data or data that your application standard is around -- it’s getting all the data all of the time.

Wood: That’s right. In some ways, this industry label of big data is perhaps not the most helpful, because it’s not just the volume of data that is the challenge and the opportunity for the business. It’s the variety of sources, as you’ve alluded to, and also the velocity at which that data is moving.

The business needs to get hold of these multiple sources of data and immediately be able to apply the analytics, get the insights, and make the business decisions. This is why still the vast majority of that data that’s available to an enterprise remains dark.

Unused and unexploited

It’s unused and unexploited. Organizations, with their traditional analytics systems, are struggling to get the meaning and insights from all these data types that we mentioned. These include unstructured information, such as social media sentiment, voice recordings, potentially even video recordings, and the structured and semi-structured things like log files and data center data. For many organizations, getting the information quickly enough out of their CRM and enterprise resource planning (ERP) systems is a challenge as well.

Gardner: So we see that there’s a great desire to do this, and there are great returns on being able to do this well. We talked about some of the general challenges. What specifically is holding people up?

Is this an issue of cost, complexity, or skills? Why aren’t companies able to move beyond this small fraction of the available information to which they could be applying such important insight and analytics?

Wood: It’s a complexity and a skills challenge, as you mentioned. The systems they have today, Dana, typically aren’t set up to able to analyze these vast amounts of unstructured information, and also to be able to analyze the structured data at a speed needed by the organization.
Typically, the analytic systems that organizations have had aren’t able to cope with that or with that unstructured human information.

Think about the need to analyze immediately a clickstream from an online shopping application or a pay-to-use application that an organization has. That is, a rapid-scale analysis of a large amount of structured data. Typically, the analytic systems that organizations have had aren’t able to cope with that or with the unstructured human information.

This is why HP has created the HAVEn Big Data Platform, and Girish will talk in more detail about this, and how it brings together the analytics engine needed to address these issues.

Just as importantly, there’s the ecosystem around HAVEn, which includes HP experts and services and services from partners, to bring together the skills needed to turn this data collection into useful information.

And there are skills around data scientists, as well -- skills around understanding the right questions the line of business needs to be asking, and understanding actually how to visualize and represent the data.

Gardner: Based on what we have talked about in terms of some of these serious challenges, Girish, what were some of the guiding principles that you were thinking of when HAVEn was being put together and refined?

Talking to customers

Mundada: HAVEn came together not by creating it in a dark room somewhere in the back office. It came together by talking to customers. On a regular basis, I meet with some of HP's largest customers worldwide, getting input from them. And they're telling us what their current problems are.

Mundada
Let me see if I can describe the landscape in a typical organization, and we can go from there. You'll see why we created HAVEn.

Let’s visualize four different waves of data. Back in early '60s,'70s, even part of the '80s, mainframes were the primary way to process data, and we used them for operationalizing certain parts of data processing, where data was extremely high-value. If you look at the cost of the systems, it was phenomenal.

Then came the next wave in the ‘80s, where we went into what I call client-server computing, and we already know several companies that were created in this space.

I’ve lived in Silicon Valley for almost 30 years now, and a whole bunch of new companies were born in this space. I worked for a company, Postgres, which became Illustra, then became Informix, and became IBM. If you look at that entire wave of OLTP technologies, we created data-processing technologies designed to solve basic business problems.

Application software was created: CRM, supplier relationship management (SRM), you name it. Many companies that did consulting around that were created, too. That was that second wave after the mainframe.
We’re talking about volumes that are growing exponentially. In the past, they were growing linearly.

Then came the third wave, where we took this data from all these transactional systems, brought them together to find out some basic analysis, which we now call business analytics, to find out "who is my most profitable customer, what are they buying, why are they buying," and things of that nature.

We created companies for that wave, too, and many technologies. Exadata, Teradata, Netezza, and a whole bunch of companies and applications were born in that space. That wave lasted for quite a while.

What we're seeing now is that from 2003 onward, something very fundamental has happened. At least, that’s the way I've been seeing this. If you look at the three Vs that Dan has described -- volume, velocity, and variety -- we’re talking about volumes that are growing exponentially. In the past, they were growing linearly. That creates a very different kind of requirement.

More importantly, if you look at the variety that Dan mentioned, that’s really the key driver in my mind. People are now routinely bringing in machine data, human data, and your traditional structured warehouses -- all of them together.

If you visualize a bar graph, you would see that 10 percent of the data that we now can monetize is coming from traditional sources, whereas 90 percent of the data that we need to monetize is now sitting in machine data and human data.

High velocity analytics

What we're trying to do with HAVEn is create a combined platform, where you can combine these three different data types and do very high-velocity analytics.

As a simple example, if you look at Apache Web Server logs, that data is used historically by the security people to see if anybody is breaking in. That data was being used by operational people to see if machines aren’t overloaded.

More importantly the digital marketing guys now want to look at that data to see who's coming to their website, what they’re buying, what they’re not buying, why they’re buying, and which geographies they’re coming from. Then, they want to combine all these data sets with their existing structured data to make sense out of it.

Today, it's a mess in the market. When we talk to our partners and customers, they’re saying that they have point solutions for each of these things, and if you want to combine that data, it’s really hard. That’s why we had to create HAVEn.

HAVEn is the fourth wave. HAVEn is specifically about big data, the fourth wave. If you look at HP’s portfolio, we sell products and services across each of these waves, and the fastest growing wave right now is the big-data wave. It’s growing at about 35 percent a year, according to Gartner, and that's why we're excited about it.
If you look at what’s required now to process big data in its entirety, one product no longer can do it all.

Gardner: Now we know why you created it and what it’s supposed to do. Tell us a little bit more about what’s included in HAVEn and why it is that you’ve been able to create a combination of product and platform that solves this very difficult task?

Mundada: If you look at what’s required now to process big data in its entirety, one product no longer can do it all. There is a very famous paper written by some university professors titled “One size does not fit all.” It proves that different data structures are able to solve different kinds of data problems far more efficiently.

One way to think about big data is to think of it as a pile of dirt. It’s a big pile. In that pile, there’s gold, silver, platinum, iron, and other metals you don’t even know. If the cost of mining that data is high, obviously you’re going to go after only the platinum and some known objects that you care about, because that’s all you can afford.

HAVEn is about bringing that cost of processing down to a very, very low level so you can go after more metals. That means you have to bring together a set of technologies to be able to solve this. If you look at the last three years, HP has made very significant amounts of investments in the big-data space.

Best of breed

We bought companies that were best of breed to try to solve specific problems. We bought Autonomy, Vertica, ArcSight, Fortify, TippingPoint, 3PAR Data, and Knightsbridge.

Now, we have a set of technologies to be able to combine them into a unique experience. Think of it almost like Microsoft Office. Before you had Microsoft Office, you would buy a word processor from one company, a spreadsheet from another company, and presentation software from a third company.

Let’s say you wanted to create a simple table. If you had created it in a word processor or even a spreadsheet, you couldn’t mix and match that. It was impossible to mix and match very different types.

Then, Microsoft came to the table and said, “Look, here’s a simplified solution.” If you want to create a table, go ahead and create it in PowerPoint. Or if you want to create more complicated thing, put it in Excel. Then, take that Excel and put it in PowerPoint. Or, you can put the whole thing into a Word document. That was the beauty of what Microsoft did.

We’re trying to do something similar for big data, make it very easy for people to combine all these different engines and the different data types and write simple applications on it.
Today you need to combine different sets of data techniques to solve different problems, and they have to work seamlessly.

Gardner: What also is going on, other than product acquisitions, is recognizing the industry standards and the H in HAVEn, being a representative of Hadoop, is an indication of that. Tell me, beyond the products, what is binding them together, and why being an open and standard space has its important role here, too?

Mundada: Let’s look at HAVEn as a platform. HAVEn is really two different concepts. There’s the HAVEn data platform, which we’ll talk about now, and there’s a HAVEn ecosystem, which I’ll mention in a minute.

HAVEn means Hadoop, Autonomy, Vertica, Enterprise Security, and “n” applications. That’s the acronym. So let’s look at one of these pieces, and why we need an architecture like this.

As I said, today you need to combine different sets of data techniques to solve different problems, and they have to work seamlessly. That’s what we did with HAVEn. I’ve been with HAVEn from day zero, before the project concept started, and I can tell you why and how we added these pieces and how we’re trying to integrate them better.

If you look at Hadoop as an ecosystem part of that HAVEn, our story with Hadoop at HP is that Hadoop is an integral part of HAVEn. We see a lot of our customers and partners betting on Hadoop and we think it’s a good thing to keep Hadoop open and non-proprietary.

Leading vendors

We also today work with all leading Hadoop vendors, so we have shipping appliances as well as reference architectures for both Cloudera and Hortonworks, and we’re working now with MapR to create similar infrastructure. That’s our Hadoop’s story.

We’ve also found that our customers are saying they want some flexibility in Hadoop. Today, they may want one vendor, and tomorrow, they may decide to go to another vendor for whatever business reasons they choose. They want to know if we can provide a simple management tool that works across multiple Hadoop distributions.

As an example, we had to extend our Business Service Management (BSM) portfolio, so we can manage Hadoop, Vertica, hardware, storage, and networking all from within one environment. This is simply operationalizing it. Having a standardized set of hardware that matches multiple Hadoop distributions was another thing we had to do. There are many such enterprise-class innovations that you’ll see coming from HP.

But more than that, we also found that Hadoop is really good for certain kinds of applications today, and obviously, the community will extend that. You will see more and more innovations coming from that community and ecosystem.
It’s an analytic database, and by that, I mean the underlying algorithms are completely designed from the ground up.

Today, there are several areas where there are holes in Hadoop, or maybe they’re not as strong as commercial products. One such area that you see is SQL. The SQL phase of Hadoop is going to be one of the key differentiators across the different Hadoop packaging.

In that area, we have a technology called Vertica, which is the V part of HAVEn, and you’ll see companies like Facebook, using a combination of both Hadoop and Vertica.

The classic use case we see is that people will bring all kinds of raw data, put it into Hadoop, and do some batch processing there. Hadoop is great as a file system, a batch processing environment. But then they’ll take pieces of that data and want to do deep analytics on it, like a regression analytics, and they will put it into Vertica.

Vertica is, is an analytic database platform, and I will break up those three words. It’s a database. It looks and feels like a database. It has SQL on it, open database connectivity (ODBC), and Java database connectivity (JDBC) connectivity. You can run all kinds of tools on it, the ones you are used to, Tableau, Pentaho, and Informatica. So from that perspective it’s a regular database.

What’s different is that it’s custom built for the fourth wave. It’s an analytic database, and by that, I mean the underlying algorithms are completely designed from the ground up. Michael Stonebraker who created the key products in the first wave and the second wave -- Ingres and Postgres -- also created this at MIT from the ground up.

Data today

The intuition was that if you look at the processing of data today, it’s gone from having 10 to 20 columns per row to possibly thousands of columns. A social media company, for example, might have 10,000 pieces of information on me, and while they do processing, it’s going more linear. It’s going regression-oriented in a sense. You might say “Girish, age x, lives here, and likes y. What’s the likelihood somebody else may like it?”

It’s meant for that kind of deep analytical processing, a column-oriented structure. In those kinds of applications, this database technology tends to be magnitudes faster -- tens of times faster. That’s one example of Hadoop and Vertica, and we can talk more about other pieces Autonomy and Enterprise Security with you.

Gardner: So we see that there’s a platform that you put together. There’s an ecosystem that’s supporting that. There are these binding standards that make the ecosystem and the platform more synergistic. But other people are doing the same thing. What’s making HAVEn different? What is it about HAVEn that you think is going to be a winner in the marketplace?

Mundada: There are two different answers to it. Let me talk about how we’ve taken just not the SQL piece of Hadoop, but how we extend it with other parts of HP that are unique to HAVEn. It’s the breadth of it. Let’s see how we extend this simple combination of Hadoop and Vertica.
With Vertica, we’re able to drop in other codes that are user defined and user written.

I said it’s an analytic database platform. If you look at that platform piece of it, with Vertica, we’re able to drop in other code that are user-defined and user-written. For example, you can drop in R language routines, Java, C++, or C language routines directly into the database. Now, we’re now able to combine that richness across our portfolio.

Autonomy, which is the A part of HAVEn, is a unique technology. It's one of a kind. Some of the largest governments and some of the largest organizations in the world, such as banks and financial institutions, have this in production in what it's meant for, human information processing, which is audio, video, and text.

As an example, you could take a video stream and ask simple questions. Tell me if an object is moving from point A to point B, or tell me what’s in the object. Is it a human? Is it a car? Can you read car number plates automatically?

And you could do some really sophisticated applications. Taking a car, we have cases where police cars have video cameras mounted on the side, and as they’re driving by in a parking lot, they can take photos of the number plates and compare it to stolen cars.


Crime detection

Imagine being able to take that technology and combining it automatically, through simple SQL-like or simple REST API-like commands with SQL, with your existing data and creating very sophisticated applications to understand your customer or for crime detection and things like that?

Now let’s bring in the third of part of the puzzle, the E part, which is Enterprise Security. That’s also unique. We have an entire portfolio, both for security as well as for operations management.

If you look at enterprise security and if you look at the Gartner Magic Quadrant, HP’s product set has been in the leader space for several years in a row. They are the number one vendor in that area.

Now, think about our portfolio of ArcSight, Fortify, Tipping Point, and other ESP products. Imagine being able to take the data-collection algorithms of those, bringing it into this common platform of HAVEn, combining it with other structured and unstructured data with just simple commands. That’s something we can do uniquely.

Operations management is another area where we have hundreds of these machine logs. We can collect them, break them open into modular pieces, and create new applications. You can go look at our website, Operations Analytics, where with a simple slider, you can go back and forth in time to millions of log files as if they were structured data.
With simple SQL, we can essentially write simple queries across structured and unstructured data.

We can do that uniquely, because we have that entire collection. Our BSM portfolio has been on the market for 30 years. It’s one of the leaders. This is the HP OpenView platform and this is one of the things we can do uniquely at HP, bring all these things together.

That’s the breadth of our portfolio, but it simply doesn’t stop at this platform level. Remember, I said that there are two concepts. There is a platform, and then there is the ecosystem. Let’s look at the platform level first.

We have the whole of HAVEn. We have the connectors, and we ship these 700 connectors out of the box. With simple commands, you can bring in social-media data in every language written. You can bring in machine logs and structured logs. That’s the platform.

Let’s extend it further into the ecosystem part. The next thing that people were saying was, “We want to use something very open. We have our own visualization tools. We have our own extract, transform, load (ETL) tools that we’re used to. Can you just make them work?" And we said, "Sure.”

That’s one of the things that we’re able to do now. With simple SQL, we can essentially write simple queries across structured and unstructured data. Using Tableau Software, or any other tool that you like, we can access this data through our connectors, but, more importantly, it let’s you hook in your existing ETL tools into this -- completely transparently.

Breadth and openness

So that’s the openness of the platform, the breadth and the openness of the platform. Breadth is not just about the software platform, but it’s about HP’s strength to bring together hardware, software, and services.

Even with the platform, the HAVEn components in the middle, the connectors, and being able to match them with matching hardware, our customers are asking, “Can you give us matching hardware for Hadoop, so we don’t have to spend time setting it up?” That’s one of things that HP can uniquely do, but more importantly we have appliances for Vertica, for example, which are standardized.

If you look at the other side, our customers are also saying, “We understand that HP wants to provide us all this, but we like openness and we like other partners.” So we said, “Fine, we’ll leave this entire ecosystem open.” Our software will work with HP hardware and we can optimize, but we also commit to working on everybody else’s hardware.
If you look at our visualization, we didn’t go and force a visualization technology on you. We kept it open.

Our cloud story is that we’ll work on Amazon, as well as OpenStack. For example, if you want to build a hybrid cloud, where part of your data resides on HP or your private environment using OpenStack, that’s fine. If you want to put it in Amazon or Rackspace, no problem. We’ll help you bridge all these. These are the kinds of enterprise-cloud innovations that HP is able to do, and we’re open to this.

So to answer your question very succinctly, if there were three things I would pick where HP is different, one is our breadth of our portfolio. We have very large breadth that we've brought together.

It’s the openness of the platform. HP is known to be a very open company. If you look our Hadoop story, we have an example. We didn’t create a proprietary Hadoop. We kept it open. If you look at our virtualization, we didn’t go and force a virtualization technology on you. We kept it open.

More importantly, if there is one key thing that you want to take home from what we've done with HAVEn, it's not about feeds and not about speeds. It's about business value.

The reason we created HAVEn was to create that iPhone-like environment or Android-like environment, where the vision is that you should be able to go to a website, say you have standardized on the HAVEn platform, and then, be able to point and click and download an application.

The end part of HAVEn is really the business value of it, and that’s how we see HAVEn as unique. There is nobody else, as far as we know, that has that end-vision, where you can build the applications yourself using standard tools -- SQL, ODBC, REST API, JDBC -- or you can buy ready-made software that HP Software has created.

We have packages across service, operations, and digital marketing. Or you can go with a partner. The partner could be HP Enterprise Services, Accenture, Capgemini, or any of those big partners. That’s something unique about the HP big-data ecosystem that doesn’t exist anywhere else today.

Applications

Gardner: Applications are something that take advantage of the platform, the capabilities, the breadth and depth of the data, and information.

I wonder if you could explain a little bit more about the application side of HAVEn, perhaps through examples of what people are already doing with these applications, and how they’re using them in their business setting?

Mundada: That’s actually one of the most exciting parts of my job. As I said, I meet literally 100 customers a month. I'm traveling across the continents, and the use-cases of big data that I see are truly phenomenal. It really keeps you very motivated to keep doing more.

Let's look at a very broad level of why these things matter. Big data is not just about monetary profits. It's really about what I call extended profits. It doesn’t have to be monetary. If you look at a simple example, we have medical companies using data, using our technologies, to dramatically speed up drug discovery hundreds of times more than they were able to with Hadoop.
HAVEn isn’t about speeds and feeds. It's about really creating business value in a hurry, so you get there before your competitors can.

That translates into just saving lives. At our Discover show, we saw that a very innovative organization is using our technology to look at bio-diversity and save wildlife in the Amazon.

That’s unique, but those are like edge cases. If you look at a regular enterprise, what they want to do at a very high level falls into three categories: Applications that HP itself is building, applications that partners are building, and applications that customers themselves are building.

Let's start with the ones that HP is building. Today HP is shipping several applications, and I’ll talk about a few of them. Even before I talk about these applications, let's look at why people generally want to do this. They’re saying that they want to either increase  revenues, so that’s affecting the top line, or they want to decrease costs, so they can increase the bottom line. Third is that they want to improve products and services. Those are really the three broad categories at a very, very high level.

As I said, HAVEn isn’t about speeds and feeds. It's about really creating business value in a hurry, so you get there before your competitors can.

From that perspective, there are three applications I’ll mention. In terms of increasing revenue, we have a product that we ship called Digital Marketing Hub, and it combines the power of Autonomy and Vertica to analyze all of your customer analytics.

You’re able to take your call center logs, your social media feeds, your emails, your phone interactions and find out what the customer is really is saying, what they want and don't want, and then, being able to optimize that interaction with the customer to create more revenue.

More precise answers

For example, when a customer calls knowing what they want, obviously you can tell them more precise things. That’s one example.

Let's look at another example, where you want to decrease your bottom line or decrease your costs. Operational Analytics is another software product we ship. We’re able to drive down costs of debugging network troubles by 80 percent by combining all these logs from machines on a very frequent basis.

We can look at this and say. "At this second, every machine was okay. A second later, machines have gone down." I can look exactly at the incremental logs that showed up, using a simple pen like a pointer, going through SQL-like data. That’s unique.

Those are the kinds of applications we’re able to create. It's not just these two. The other thing people want is improve products and services. We have something called Service Anywhere, where as you're calling or as you're typing in commands and saying you want to find information about that, the system is able to understand the meaning of what you’re saying.

Notice that this is not keyword search. This is meaning, where it's able to go through existing case reports from customers, look at existing resolutions, and then say, “Okay, this might solve your problem automatically.”
That’s the beauty of the HAVEn platform. On the same platform, you can buy HP built applications or you can build your own.

Imagine what that impacts. Your customers are happy, because the answers are quicker. We call this ticketless ID, but more important, look at some other interesting ways of how this affects a company.

For example, I was recently in Europe. I was talking to a very large telco there, and they said, “We have something like 20,000 call-center operators who are taking calls from customers. Each call volume might take six minutes and some of them are repeat calls. That’s really our problem.”

We worked out something that roughly could save them two minutes per call. That translates to about a $100 million net saving per year. That’s really phenomenal. Those are one kind of application that HP built.

Now imagine a customer wanting to build the same application themselves. That’s the beauty of the HAVEn platform. On the same platform, you can buy HP built applications or you can build your own.

Let's look at NASCAR as an example. They did something very similar for customer analytics. They are able to -- while the race is happening -- understand audio, television channels, radio, broadcast, and social media and bring that all together as if it's one unique piece of data.

Then, they’re able to use that data in really innovative ways to further their sport and to create more promotional dollars for just not themselves, but even the participants. That’s unique -- being able to analyze mass scale human data.

Looking to the future

Gardner: Well, we've learned a lot about the market, the demand, why big data makes so much sense. There is very large undertaking by HP around HAVEn, and what it’s getting in terms of openness, platforms, breadth, and these great examples of applications. But we also need to look to the future.

What's coming next in terms of HAVEn 2.0 or HAVEn 1.5? Dan, could you update us on how things are progressing, what you have in mind for the next versions of these products and, therefore, the whole increasing as sum of the parts increases?

Wood: Dana, we've just announced HAVEn 2.0. The way Girish explained HAVEn there in terms of the platform and the ecosystem and continuous innovation now is around both of those pieces. It's really important to us to be driving the ecosystem, as well as the platform. So I’ll speak to HAVEn 2.0 and one of the feature that’s the focus in driving HP forward.

In terms of the platform, there are the analytics engines that we have. Girish mentioned they were best in class at the time that HP acquired them, and we continue to invest in R and D across Autonomy, IDOL, Vertica, and the ArcSight Logger product. We recently announced new versions of all three of those, improving the analytics capability and the usability and, just as importantly, increasing the interoperability.
At the moment, on an early-access program, we’re making the IDOL engine available to developers as a cloud-based offering.

For example, we now have integration of the ArcSight Logger with the Autonomy IDOL engine for analyzing unstructured human information. A really great use case of this is Logger was previously enabling IT to understand data movements and potential threats and the risks in the organization.

For example, if I were sending 50 percent of my email to a competitor, you could combine that capability with the unstructured information analysis in Autonomy and understand by that the information layer exactly what’s in that email, 50 percent of which is going to a competitor.

Let’s start putting that together and getting a powerful view of what an individual is doing and whether it’s a risky individual in the organization, integrating those HAVEn engines and putting more effort on integrating it into the Hadoop environment as well.

For example, we have just announced integration Hadoop connectors for Autonomy. A lot of people are saying that they’re building this data lake with Hadoop and they want to have the capability of putting some analytics into the unstructured information that exists in that Hadoop data lake. Clearly, we’ve also got integration with Vertica in the Hadoop environment as well.

The other key thing within that on the engine is IDOL OnDemand. At the moment, on an early-access program, we’re making the IDOL engine available to developers as a cloud-based offering. This is to encourage the independent developer community to take components of IDOL with that social media analytics, whether it’s video or audio recognition, and start building that into their own applications.

We believe the power of HAVEn will come from the combination of HP-provided applications and also third-party applications on top.

Early-access program

We’re facilitating that with this initial early-access program on IDOL OnDemand, and also, we’re investing in developer programs to make the whole HAVEn development platform far easier for partners and independent developers to work with.

We’ve set up a HAVEn developer website, and stay tuned for some really fun events online and physical events, where we’ll be getting the developer community together.

In terms of those applications that make the whole HAVEn ecosystem come to life, Girish has mentioned some of them that we have announced over the last few weeks. So I’ll give you a quick recap on those.

We have the Operations Analytics and Service Anywhere apps, both aimed at the CIO. And we have the Digital Marketing Hub from HP aimed at marketing leaders in the organizations. These are three applications that HP has packaged on the HAVEn platform.

And along with the HAVEn 2.0 announcement, we’re really pleased that six of the leading SI partners -- Accenture, Capgemini, Deloitte, PwC, Accenture and Wipro -- themselves have put marketing applications on top of HAVEn. And those guys have gotten fascinating mixtures of very industry-specific analytics applications and more horizontal apps based on the priorities that they’re chasing after.
We’re populating our solutions and partner solutions to facilitate the whole commerce side of those applications taking off in the market.

So we’re really excited about that and expect to see many more announcements of partner applications over the next few months.

The final piece of HAVEn 2.0 to support this whole ecosystem thing is a marketplace that we’ve launched, where we’re populating our solutions and partner solutions to facilitate the whole commerce side of those applications taking off in the market.

Gardner: Just to flesh out that last point, when you say a marketplace, is this an app store? Will some of your partners that are able to create analytics-oriented applications on HAVEn then be able to sell them? Is this a commerce site or is it a community site only at this point?

Mundada: The original vision of HAVEn was to be able to make it essentially like how you buy applications on a mobile phone today. Once you have settled on a platform, the eventual vision is to be able to go there and just download these applications. As Dan said, they’ve launched this now and you will see much more stuff coming in this area.

Gardner: For those interested in learning more, those who might want to focus on one element of HAVEn, it’s not all-inclusive. You don’t have to buy it all at once. It comes in parts. There are on-ramps, and then you can expand. How do you get started? How do you learn about specific parts of HAVEn? Which combinations would work for you?

One-stop resource

Wood: The first place to go is hp.com/haven. That’s your one-stop resource for information on this platform, all of the engines that Girish alluded to. You can get the inspiration from some amazing customer case studies we have on there -- insights from experts like Girish and other people who are talking in depth about the individual engines.

And as you rightly say, Dana, it’s finding the right on-ramp for yourself.  You can look at the case studies we have, the use cases on big data in particular industries, and take a look at what the specific pain point you have today. That’s the hp.com/haven website, and that gives you all of that information.

You can also drill down from there, if you're a developer, and find the tools and resources that we’ve spoken about to enable you to start building apps on top of HAVEn. That’s one part.

The whole power of HP behind this HAVEn platform is in enabling, from an infrastructure and services point of view, to start building these big data analytics. A couple of key things here.

We started to build fully configured appliances around Hadoop and Vertica. So the Converged System’s team in HP has launched the ConvergedSystem 300, which enables you to have Vertica and Hadoop on a pre-configured appliance. That’s a great starting point for someone early on in the big-data analytics life cycle.
Those guys have data scientists and industry experts who can actually help customers go through the design phase for a big-data platform.

To expand on that, the Technology Services team is able to do full consulting on how to optimize the overall infrastructure from the point of view of processing, sharing, and storing this vast amount of information that all organizations are coping with today. That will then start to put in things like 3PAR storage systems and other innovations across the HP hardware business.

Another place where I see customers often needing some help to get started is in understanding exactly what the questions are that we need to be asking in terms of analytics and exactly what algorithms and analytics we need to put in place to get going. This is where the Big Data Discovery Experience Services from HP come in.

This is provided by the Enterprise Services Group (ESG). Those guys have data scientists and industry experts who can actually help customers go through the design phase for a big-data platform and than offer the HAVEn infrastructure supported by the ESG Services team.

Finally, Dana, come and see us on the road. We’ll be at HP Discover in Las Vegas June 10-12. We’re putting together several road shows and events across the main regions in Europe, the Americas, and in Asia Pacific, where we will be taking HAVEn on the road, too. Take a look at that hp.com/haven website, and details of the events will be found on there.

Key messages

Mundada: I wanted to add a couple of things that are a bit relevant. There are two key messages: big data is really important and it’s disrupting business. Your competitors are going to do it. You have a choice to either lead and do it yourself or you will be forced to follow. It’s one of those things that are disrupting industries worldwide.

Now, when you think of big data, don’t think of pieces and don’t think of piece parts. It’s not like you need a separate solution for human information, another for machine logs, and another for structured data. You almost have to think of it holistically, because there are many kinds of newer applications that I’m seeing regularly, where you have to bring all these data types together and create joint applications.

Whichever technologies that you choose and settle on, think of that Microsoft Office-like experience. You want to combine integrated solution across the entire stack and there aren’t that many available in the market today. So whoever you work with, make sure that you’re able to handle that entire piece as one giant puzzle.
You want to combine integrated solution across the entire stack and there aren’t that many available in the market today.

Gardner: Very good. I’m afraid we we'll have to leave it there. You’ve been listening to an executive-level discussion highlighting how the latest version of HP HAVEn produces new business analytics value and strategic return. We have seen how big-data capabilities and advanced business analytics have now really become essential to nearly any business activity.

This discussion marks the latest episode in the ongoing HP Big Data Podcast Series, where leading edge adopters of data-driven business strategies share their success stories and where the transformative nature of big data takes center stage.

Please join me now in thanking today’s guests, Girish Mundada, Chief Technology Officer for HP HAVEn. Thanks so much, Girish.

Mundada: Thanks.

Gardner: And also Dan Wood, Worldwide Solution Marketing Lead for Big Data at HP Software. Thank you, Dan.

Wood: Thanks, Dana.

Gardner: To learn more about how businesses anywhere can best capture knowledge, gain deeper analysis, and rapidly and securely make those insights available to more people on their terms, visit the HP HAVEn Resource Center at hp.com/haven.

I’m Dana Gardner, Principal Analyst at Interarbor Solutions, your moderator for this ongoing sponsored journey into how data is analyzed and used to advance the way we live and work. Thanks so much for listening and do come back next time.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect podcast on how HP is developing products and platforms to help businesses deal with the demands of big data in a competitive environment. Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in:

Friday, March 07, 2014

Fast-Changing Demands on Data Centers Drive Need for Automated Data Center Infrastructure Management

Transcript of a BriefingsDirect discussion on how organizations need to better manage the impact that IT and big data now have on data centers and how Data Center Infrastructure Management helps.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Dana Gardner: Hi, this is Dana Gardner, Principal Analyst at Interarbor Solutions, and you’re listening to BriefingsDirect.

Gardner
Today, we present a sponsored podcast discussion on improving the management and automation of data centers. As data centers have matured and advanced to support unpredictable workloads like hybrid cloud, big data, and mobile applications, the ability to manage and operate that infrastructure efficiently has grown increasingly difficult.

At the same time, as enterprises seek to rationalize their applications and data, centralization and consolidation of data centers has made their management even more critical -- at ever larger scale and density.

So how do enterprise IT operators and planners keep their data centers from spinning out of control despite these new requirements? How can they leverage the best of converged systems and gain increased automation, as well as rapid analysis for improving efficiency?

We’re here to pose such questions to two experts from HP Technology Services, and thereby explore how new integrated management capabilities are providing the means for better and automated data center infrastructure management (DCIM).

Here now to explain how disparate data center resources can be integrated into broader enterprise management capabilities and processes, we’re here with Aaron Carman, HP Worldwide Critical Facilities Strategy Leader. Welcome to BriefingsDirect, Aaron. [Learn more about DCIM.]

Aaron Carman: It's pleasure to be here. Thank you.

Gardner: We’re also here with Steve Wibrew, HP Worldwide IT Management Consulting Strategy and Portfolio Lead. Welcome, Steve.

Steve Wibrew: Hello, and glad to be here. Thank you.

Gardner: Aaron, let me start with you. From a high level, what’s forcing these changes in data center management and planning and operations? What are these big new requirements? Why is it becoming so difficult?

Carman: It's a very interesting question that people are actually trying to deal with. What it comes down to is that in the past, folks were dealing with traditional types of services that were on a traditional type of IT infrastructure.

Standard, monolithic-type data centers were designed one-off. In the past few years, with the emergence of cloud and hybrid service delivery, as well as some of the different solutions around convergence like converged infrastructures, the environment has become much more dynamic and complex.

Hybrid services

So, many organizations are trying to grapple with, and deal with, not only the traditional silos that are in place between facilities, IT, and the business, but also deal with how they are going to host and manage hybrid service delivery and what impact that’s going to have on their environment.

Carman
It’s not only about what the impact is going to be on rolling out new infrastructure solutions like converged infrastructures from multiple vendors, but how to increasingly provide more flexibility and services to their end users as digital services.

It's become much more complex and it's a little bit harder to manage, because there are many, separate types of tools that they use to manage these environments, and it has continued to increase.

Gardner: Steve, do you have anything more to offer in terms of how the function of IT is changing? I suppose that with ITIL v3 and more focus on a service-delivery model, even the goal of IT has changed.

Wibrew: That's very true. We’re seeing a trend in the change and role of IT to the business. Previously IT was a cost center, an overhead to the business, to deliver the required services. Nowadays, IT is very much the business of an organization, and without IT, most organizations simply cease to function. So IT, its availability and performance, is a critical aspect of the success of the business.

Gardner: What about this additional factor of big data and analysis as applied to IT and IT infrastructure. We’re getting reams and reams of data that needs to be used and managed. Is that part of what you’re dealing with as well, the idea that you can be analyzing in real-time what all of your systems are doing and then leverage that?

Wibrew
Wibrew: That’s certainly a very important part of the converged-management solution. There’s been a tremendous explosion in the amount of data, the amount of management information, that's available. If you narrow that down to the management information associated with operating management and supporting data centers from the facility to the applications, to the platforms right up to the services to the business, clearly that's a huge amount of information that’s collected or maintained on a 24×7 basis.

Making good and intelligent decisions on that is quite a challenge for many organizations. Quite often, we would be saying that people still remain in isolated silo teams without good interaction between the different teams. It's a challenge trying to draw that information together so businesses can make intelligent choices based on analytics of that end-to-end information.

Gardner: Aaron, I’ve heard that word "silo" now a few times, siloed teams, siloed infrastructure, and also siloed management of infrastructure. Are we now talking about perhaps a management of management capabilities? Is that part of your story here now?

Added burden

Carman: It is. For the most part, most organizations when faced with trying to manage these different areas, facilities IT and service delivery, have come up with their own set of run books, processes, tools, and methodologies for operating their data center.

When you put that onto an organization, it's just an added burden for them to try to get vendors to work with one another and integrate software tools and solutions. What the folks that provide these solutions have started to realize is that there needs to be an interoperability between these tools. There has never really been a single tool that could do that, except for what has just emerged in the past few years, which is DCIM.

HP really believes that DCIM is a foundational, operational tool that will, when properly integrated into an environment, become the backbone for operational data to traverse from many of the different tools that are used to operate the data center, from IT service management (ITSM), to IT infrastructure management, and the critical facilities management tools.

Gardner: I suppose yet another trend that we’re all grappling with these days is the notion of things moving to as-a-service, on-demand, or even as a cloud technology. Is that the case, too, with DCIM, that people are looking to do this as a service? Are we starting to do this across the hybrid model as well?
Today, clients have a huge amount of choice in terms of how they provision and obtain their IT.

Carman: Yes. These solution providers are looking toward how they can penetrate the market and provide services to all different sizes of organizations. Many of them are looking to a software-as-a-service (SaaS) model to provide DCIM. There has to be a very careful analysis of what type of a licensing model you're going to actually use within your environment to ensure that the type of functionality you're trying to achieve is interoperable with existing management tools.

Gardner: Steve, do you have anything more to offer in terms of where this is going, perhaps over time on that services delivery question? [Learn more about DCIM.]

Wibrew: Today, clients have a huge amount of choice in terms of how they provision and obtain their IT. Obviously, there are the traditional legacy environments and the converged systems and clients operate in their own cloud solutions.

Or maybe they’re even going out to external cloud providers and some interesting dynamics that really do increase the complexity of where they get services from. This needs to be baked into that converged solution around the interoperability and interfacing between multiple systems. So IT is truly a business supporting the organization and providing end-to-end services.

Gardner: Well I can certainly see why IDC recently named 2014 is the year of DCIM. It seems that the timing now is critical. If you let your systems languish in legacy status for too long, you won’t be able to keep up with the new demand. If you don’t create management-of-management capabilities, you won’t be able to cross these boundaries of service delivery and hybrid models and you certainly won’t be able to exploit the analysis change from all the data.

So it seems to me that this is really the time to get on this before you lose ground and/or can’t keep up with the modern requirements. What’s happening right now in terms of HP and how it’s trying to help organizations obtain do some sooner rather than later? Let me start with you, Aaron.

Organizations struggling

Carman: Most organizations are really struggling to introduce DCIM into their environment, since at this point, it’s really viewed as more as a facilities-type tool. The approach from different DCIM providers varies greatly on the functions and features they provide in their tool. Many organizations are struggling just to understand which DCIM product is best for them and how to incorporate into a long term strategy for operations management.

So the services that we brought to market address that specifically, not only from which DCIM tool will be best for their environment, but how it fits strategically into the direction they want to take from hosting their digital services in the future.

Gardner: Steve, I think we should also be careful not to limit the purview of DCIM. This is not just IT. This does include facilities, hybrid and service delivery model, management capabilities. Maybe you could help us put the proper box around DCIM. How far and why does it go or should we narrow it so that it doesn’t become deluded or confused?

Wibrew: Yeah, that’s a very good question, an important one to address. What we’ve seen is what the analysts have predicted. Now is the time, and we’re going to see huge growth in DCIM solutions over the next few years.
DCIM alone is not the end-to-end solution.

DCIM has really been the domain of the facilities team, and there’s traditionally been quite a lack of understanding of what DCIM is all about within the IT infrastructure management team. If you talk to lot of IT specialists, the awareness of DCIM is still quite limited at the moment. So they certainly need to find out more about it and understand the value that DCIM can bring to IT infrastructure management.

I understand that features and functions do vary, and the extent of what DCIM delivers will vary from one product to another. It’s very good certainly around the facilities space in terms of power, cooling, and knowing what’s out on the data center floor. It’s very good at knowing what’s in the rack and how much power and space has been used within the rack.

It’s very good at cable management, the networks, and for storage and the power cabling. The trend is that DCIM will evolve and grow more into the IT management space as well. So it’s becoming very aware of things like server infrastructure and even down to the virtual infrastructure, as well, getting into those domains.

DCIM will typically have work protectabilities for change in activity management. But DCIM alone is not the end-to-end solution, and we realized the importance of the need to integrate it with the full ITSM solutions and platform management solutions. A major focus, over the past few months, is to make sure that the DCIM solutions do integrate very well with the wider IT service-management solutions to provide that integrated end-to-end holistic management solution across the entire data-center ecosystem.

Gardner: Aaron, when I hear Steve talking about this more general inclusion description of DCIM, it occurs to me that this isn’t something you buy in a box. This is not just a technology or a product that we’re talking about. We’re talking about methodology. We’re talking about consulting, expertise, and tribal knowledge that’s shared. Maybe you could help us better understand not only HP’s approach to this, but how one attains DCIM. What is the process by which one becomes an expert in this? [Learn more about DCIM.]

Great variation

Carman: With DCIM being a newer solution within the industry, I want to be very careful about calling folks DCIM specialists. We feel that we have a very great knowledge of the solutions out there. They vary so greatly.

It takes a collaborative team of folks within HP, as well as with the client, to truly understand what they’re trying to achieve. You could even pull it down to what types of use cases they’re trying to achieve for the organization, which tool works best and in interoperability and coordination with the other tools and processes they have.

We have a methodology framework called the Converged Management Framework that focuses on four distinct areas for a optimized solution and strategy for starting with business goals and understanding what the true key performance indicators are and what dashboards are required.

It looks at what the metrics are going to be for measuring success and couples that with understanding organizationally who is responsible for what types of services we provide as an ultimate service to our end user. Most of the time, we’re focusing on the facilities in IT organization.

Also, those need to be aligned to the process and workflows for provisioning services to the end users, supported directly by a system’s reference architecture, which is primarily made up of operational management tools and software. All those need to be supported by one another and purposefully designed, so that you can meet and achieve the goals of the business.
IT infrastructure, right up to services of a business, end to end, is very large and very, very complex.

When you don’t do that, the time it takes for you to deliver services to your end user lengthens and costs money. When you have separate tools that are not referencing single points of data, then you’re spending a lot of time rationalizing and understanding if you have the accurate data in front of you. All this boils down to not only cost but having a resilient operations, knowing that when you’re looking at a particular device or setup devices, you truly understand what it’s providing end to end to your users.

Gardner: Steve, it seems to me that this is a little bit of a chameleon. People who have a certain type of requirement can look at DCIM, some of the methodologies and framework, and get something unique or tailored.

If someone has real serious energy issues, they’re worried about not being able to supply sufficient energy. So they could approach DCIM from that energy vantage point. If someone is building a new data center, they could bring facilities planning together with other requirements and have that larger holistic view.

Am I reading this right? Is this sort of a chameleon or an adaptive type of affair, and how does that sort of manifest itself in terms of how you deliver the service?

Wibrew: If you think about the possibilities in the management of facilities, the IT infrastructure, right up to services of a business, end-to-end, is very large and very, very complex. We have to break it down into small or more manageable chunks and focus on the key priorities.

Most-important priorities

So we look at the trans-organization, work with them to identify to them what their most important priorities are in terms of their converged-management solution and their journey.

It’s heavily structured around ITSM and ITIL processes, and we’ve identified some great candidates within ITIL for integration between facilities in IT. It’s really a case of working out the prioritized journey for that particular client. Probably one of the most important integrations would be to have a single view of the truth of operational data. So it would be unified asset information.

CMDBs within a configuration management system might be the very first and important integration between the two, because that’s the foundation for other follow-on services until you know what you’ve got, it’s very difficult to plan, what you need in the future in terms of infrastructure.

Another important integration that is now possible with these converged solutions is the integration of power management in terms of energy consumption between the facilities and the IT infrastructure.
These integrated solutions can be more granular, far more dynamic around energy consumption.

If you think about managing the power consumption of things like efficiency of the data center with PoE, generally speaking, in the past, that would be the domain of the facilities team. The IT infrastructure would simply be hosted in the facility.

The IT teams didn’t really care about how much power was used. But these integrated solutions can be more granular, far more dynamic around energy consumption with much more information being collected, not just at a facility level but within the racks and in the power-distribution units (PDUs), and in the blade chassis, right down to individual service.

We can now know what the energy consumption is. We can now incentivize the IT teams to take responsibility for energy management and energy consumption. This is a great way of actually reducing a client’s carbon foot print and energy consumption within the data center through these integrated solutions.

Gardner: Aaron, I suppose another important point to be clear on is that, like many services within HP Technology Services, this is not just designed for HP products. This is an ecumenical approach to whatever is installed in terms of product facility management capability. I wonder if you could explain a bit more HP’s philosophy when it comes to supporting the entire portfolio. [Learn more about DCIM.]

Carman: HP’s professional services we’re offering in this space are really agnostic to the final solution. We understand that a customer has been running their environment for years and has made investments into a lot of different operational tools over the years.

That’s a part of our analysis and methodology, to come in and understand the environment and what the client is trying to achieve. Then we put together a strategy, a roadmap of different products, that will help them achieve their goals that are interoperable.

Next level

We continue to transform them to the next level of abilities or capabilities that they are looking to achieve, especially around how they provision services and help them become, at the end, most likely a cloud-service provider to their end users, where heavy levels of automation are built in, so that they can get digital services to their end users in a much shorter period of time.

Gardner: One of the things I really like in talking about technology is to focus on the way it’s being used, to show rather than just tell. I’m hoping that either of you, Aaron or Steve, have some use cases or examples where this has been put to good use -- DCIM processes, methodologies, the über-holistic approach, and planning right down to the chassis included.

I hope you can not only discuss a little bit about by who and how this is being done, but what they get for it. Are there any data points we can look to that tell us that, when people do this right -- and here are some folks that have done it right -- what they got back for their efforts. Why don’t we start with you, Aaron?

Carman: HP has been offering operational services for years. So this is nothing new to HP, but traditionally, we’ve been providing these services in silos. When we reorganized ourselves just recently and really started to put together the IT-plus-facilities store, it quickly became very apparent that from an operations management perspective, a lot of the services we provide really needed to have a lifecycle approach and be brought together.

So we have a lot of different examples. We’ve rolled out different forms of converged-management consulting to other clients, and there are a lot of different benefits you get from the different tools that are a part of the overall solution.
We’re providing folks with a means of optimizing how they provision services, which is going to lower their cost structures.

You can point to DCIM and a lot of the benefits you get from understanding your assets and being able to decommission those more quickly, understanding the power relationship, and then understanding many different elements of tying the IT infrastructure chain to the facilities chain.

In the end, when you look at all these together, it’s going to be different for every client. You have to come in and understand the different components that are going to make up a return on investment (ROI) for the client based upon what they’re willing to do and what they’re trying to achieve.

In the end, we’re providing folks with a means of optimizing how they provision services, which is going to lower their cost structures. Everyone is looking to lower cost, but also increase resiliency, as well as then possibly defer large capital expenditures like expanding data centers. So many of these different outcomes could apply to a customer that engages with converged management.

Gardner: I realize this is fairly new. It was just on Jan. 23 that HP announced some newservices that are converged-management consulting, and that management framework was updated with new technical requirements. You have four new services organized with the management workshop, roadmap, design implementations, and so forth. [Learn more about DCIM.]

So this is fairly new, but Steve Wibrew, is there any instance where you’ve worked with some organization and that some of the really powerful benefits of doing this properly have shown through? Do you have any anecdotes you can recall of an organization that’s done this and maybe some interesting ways that it’s benefited them, maybe unintended consequences?

Data-center transformation

Wibrew: I certainly can give some real examples. Where I've worked in the past with some major projects for transformation within the data center, we would be deploying large amounts of new infrastructure within the data center.

The starting point is to understand what’s there in the first place. I’ve been engaged with many clients where if you ask them about inventory, what’s in the data center, you get totally different answers from different groups of people within the organization. The IT team wants to put more stuff into the data center. The facilities team says, “No more space. We’re full. We can’t do that.”

I found that when you pull this data together from multiple sources and get a consistent feel of the truth, you can start to plan far more accurately and efficiently. Perhaps the lack of space in the data center is because there may be infrastructure that’s sitting there, powered on, and not being utilized by anybody.

It’s a fact that we’re redundant. I’ve had many situations where, in pulling together a consistent inventory, we can get rid of a lot of redundant equipment, allowing space for major initiatives and expansion projects. So there are some examples of the benefits of consolidated inventory and information.
DCIM is the only tool poised to become that backbone between the facilities and IT infrastructures.

Gardner: We’re almost out of time, but I just wanted to look towards the future about the requirements and the dynamic nature of workloads and the scale and density of consolidated data centers. I have to imagine that these are only going to become more urgent and more pressing.

So what about that, Aaron, as we look a few years out at big-data requirements, hybrid cloud requirements, infrastructure KPIs for service delivery, energy, and carbon pressures? What’s the outlook in terms of doing this, and should we expect that there will be an ongoing demand, but also ongoing and improving return on investments you make, vis-à-vis these consulting services and DCIM?

Carman: Based upon a lot of the challenges that we outlined earlier in the program, we feel that in order to operate efficiently, this type of a future state operational-tools architecture is going to have to be in place, and DCIM is the only tool poised to become that backbone between the facilities and IT infrastructures.

So more-and-more, with a lot of the challenges of my compute footprint shrinking and having a different requirements that I had in the past, we’re now dealing with a storage or data explosion, where my data center is all filled up with storage files.

As these new demands from the business come down and force organizations onto new types of technology infrastructure platforms they haven’t dealt within the past, it requires them to be much more flexible when they have, in most cases, very inflexible facilities. That’s the strength of DCIM and what it can provide just in that one instance.

But more-and-more, the business is expecting digital services to almost be instant. They want to capitalize on the market at that time. They don't want to wait weeks or months for enterprise IT to provide them with a service to take advantage of a new service offering. So it's forcing folks into operating differently, and that's where converged management is poised to help these customers.

Looking to the future

Gardner: Last word to you, Steve. When you look into your crystal ball and think about how things will be in three to five years, what is it about DCIM rather and some of these services that you think will be most impacting?

Wibrew: I think the trend we're going to see is a far greater adoption of DCIM. It's only deployed in a small number of data centers at the moment. That's going to increase quite dramatically, and this could be a much tighter alignment between how the facilities are run and how the IT infrastructure is operated and supported. It could be far more integrated than it is today.

The roles of IT are going to change, and a lot of the work now is still around design, planning, scripting, and orchestrating. In the future, we're going to see people, almost like a conductor in an orchestra, overseeing the operations within the data center through leading highly automated and optimized processes, which are actually delivered by automated solutions.
The trend we're going to see is a far greater adoption of DCIM. It's only deployed in a small number of data centers at the moment.

Gardner: Very good. I should also point out that I benefited greatly in learning more about DCIM on the HP website. There were videos, white-papers, and blog-posts. So, there’s quite a bit of information for those interested in learning more about DCIM. HP Technology Services website was a great resource for me. [Learn more about DCIM.]

We'll have to leave it there, gentlemen. You’ve been listening to a sponsored BriefingsDirect discussion on improving the management and automation of data centers and facilities. We’ve seen how IT operators and planners can keep their data centers from spinning out of control via exploiting new data-center infrastructure management capabilities.

I want to thank our guests, Aaron Carman, the HP Worldwide Critical Facilities Strategy Leader. Thanks so much, Aaron.

Carman: It's my pleasure. Thank you.

Gardner: And also Steve Wibrew, HP Worldwide IT Management Consulting Strategy and Portfolio Lead. Thanks so much, Steve.

Wibrew: Thank you for listening.

Gardner: This is Dana Gardner, Principal Analyst at Interarbor Solutions. Thanks to our audience and come back next time for the next BriefingsDirect podcast discussion.

Listen to the podcast. Find it on iTunes. Download the transcript. Sponsor: HP.

Transcript of a BriefingsDirect discussion on how organizations need to better manage the impact that IT and big data now have on data centers and how Data Center Infrastructure Management helps. Copyright Interarbor Solutions, LLC, 2005-2014. All rights reserved.

You may also be interested in: