What can GPFS on Hadoop do for you ?

The Hadoop Distributed File System (HDFS) is considered a core component of Hadoop, but it’s not an essential one. Lately, IBM has been talking up the benefits of hooking Hadoop up to the General Parallel File System (GPFS). IBM has done the work of integrating GPFS with Hadoop. The big question is, What can GPFS on Hadoop do for you?

IBM developed GPFS in 1998 as a SAN file system for use in HPC applications and IBM’s biggest supercomputers, such as Blue Gene, ASCI Purple, Watson, Sequoia, and MIRA. In 2009, IBM hooked GPFS to Hadoop, and today IBM is running GPFS, which scales into the petabyte range and has more advanced data management capabilities than HDFS, on InfoSphere BigInsights, its collection of Hadoop-related offerings, as well as Platform Symphony.

GPFS was originally developed as a SAN file system. That would normally prevent it from being used in Hadoop and the direct-attach disks that make up a cluster. This is where an IBM GPFS feature called File Placement Optimization (FPO) comes into play.

Phil Horwitz, a senior engineer at IBM’s Systems Optimization Competency Center, recently discussed how IBM is using GPFS with BigInsights and System x servers, and in particular how FPO has is helping GPFS to make inroads in a Hadoop cluster. (IBM has since sold off the System x business to Lenovo, which IBM now must work closely with for GPFS-based solutions, but the points are still valid).

According to Horwitz, FPO essentially emulates a key component of HDFS: moving the application workload to the data. “Basically, it moves the job to the data as opposed to moving data to the job,” he says in the interview. “Say I have 20 servers in a rack and three racks. GPFS FPO knows a copy of the data I need is located on the 60th server and it can send the job right to that server. This reduces network traffic since GPFS- FPO does not need to move the data. It also improves performance and efficiency.”

Last month, IBM published an in-depth technical white paper titled “Deploying a Big Data Solution using IBM GPFS-FPO” that explains how to roll out GPFS on Hadoop. It also explains some of the benefits users will see from using the technology. For starters, GPFS is POSIX compliant, which enables any other applications running atop the Hadoop cluster to access data stored in the file system in a straightforward manner. With HDFS, only Hadoop applications can access the data, and they must go through the Java-based HDFS API.

The flexibility to access GPFS-resident data from Hadoop and non-Hadoop applications frees users to build more flexible big data workflows. For example, a customer may analyze a piece of data with SAS. As part of that workflow, they may use a series of ETL steps to manipulate data. Those ETL processes may be best executed by a MapReduce program. Trying to build this workflow on HDFS would require additional steps, as well as moving data in and out of HDFS. Using GPFS simplifies the architecture and minimizes the data movement, IBM says.

There are many other general IT housekeeping-type benefits to using GPFS. According to IBM’s "Harness the Power of Big Data" publication, POSIX compliance also allows users to manage their Hadoop storage “just as you would any other computer in your IT environment.” This allows customers to use traditional backup and restore utilities with their Hadoop clusters, as opposed to using the “copy” command in HDFS. What’s more, GPFS supports point-in-time snapshots and off-site replication capabilities, which aren't available in plain-vanilla HDFS.

The size of data blocks is also an issue with HDFS. In IBM's June 2013 whitepaper "Extending IBM InfoSphere BigInsights with GPFS FPO and IBM Platform Symphony" IBM makes the case that, because Hadoop MapReduce is optimized for blocks that are around 64MB in size, HDFS is inefficient at dealing with smaller data sizes. In the world of big data, it's not always the size of the data that matters; the number of data points and the frequency at which the data changes is important too.

GPFS also brings benefits in the area of data de-duplication, because it does not tend to duplicate data as HDFS does, IBM says. However, if users prefer to have copies of their data spread out in multiple places on their cluster, they can use the Write-affinity depth (WAD) feature that debuted with the introduction of FPO. The GPFS quote system also helps to control the number of files and the amount of file data in the file system, which helps to manage storage.

Capacity planning of Hadoop clusters is easier when the data stored in GPFS, IBM says. In HDFS, administrators need to carefully design the disk space dedicated to the Hadoop cluster, including dedicating space for the output of MapReduce jobs and log files. “With GPFS-FPO,” IBM says, “you only need to worry about the disks themselves filling up; there’s no need to dedicate storage for Hadoop.”

Other benefits include the capability to used policy-based information lifecycle management functions. That means third-part management tools, such as IBM’s Tivoli Storage Manager software, can manage the data storage for internal storage pools. The hierarchical storage management (HSM) capabilities that are built into GPFS mean you can keep the “hottest” data on the fastest disks. That feature is not available in plan-vanilla Hadoop running HDFS.

The shared-nothing architecture used by GPFS-FPO also provides greater resilience than HDFS by allowing each node to operate independently, reducing the impact of failure events across multiple nodes. The elimination of the HDFS NameNode also eliminates the single-point-of-failure problem that shadows enterprise Hadoop deployments. “By storing your data in GPFS-FPO you are freed from the architectural restrictions of HDFS,” IBM says.

The Active File Management (AFM) feature of GPFS also boosts resiliency by caching datasets in different places on the cluster, thereby ensuring applications access to data even when the remote storage cluster is unavailable. AFM also effectively masks wide-area network latencies and outages. Customers can either use AFM to maintain an asynchronous copy of the data at a separate physical location or use GPFS synchronous replication, which are used by FPO replicas.

Security is also bolstered with GPFS. Customers can use either traditional ACLs based on the POSIX model, or network file system (NFS) version 4 ACLs. IBM says NFS ACLs provide much more control of file and directory access. GPFS also includes immutability and appendOnly restriction capabilities, which can be used to protect data and prevent it from being modified or deleted.

You don’t have to be using IBM’s BigInsights (or its Platform Symphony offering) to take advantage of GPFS. The company will sell the file system to do-it-yourself Hadoopers, as well as those who are running distributions from other companies. And using GPFS allows you to use the wide array of Hadoop tools in the big data stack, such as Flume, Sqoop, Hive, Pig, Hbase, Lucene, Oozie, and of course MapReduce itself.

IBM added the FPO capabilities to GPFS version 3.5 in December 2012. Although it's POSIX compliant, GPFS-FPO is only available on Linux at this point. IBM says GPFS is currently being used in a variety of big data applications in the areas of bioinformatics, operational analytics, digital media, engineering design, financial analytics, seismic data processing, and geographic information systems.

By : Alex Woodie
Link :http://www.datanami.com/datanami/2014-02-18/what_can_gpfs_on_hadoop_do_for_you_.html

This tiny chip makes the Internet four times faster

 

 

The race is on to build a faster, better Internet. While Google is working on bringing super-high-speed connections to homes in select cities, IBM is working on a technology that could make the Internet all around faster everywhere.

It has created a new chip that beefs up Internet speeds to 200 to 400 gigabits per second, about four times faster than today's speeds, IBM says. Plus it sucks up hardly any power.

At this speed, a 2-hour ultra-high-definition movie (about 160 gigabytes) would download in a few seconds. It would only take a few seconds to download 40,000 songs, IBM says.

The chip fits into a part of the Internet that runs between data centers, not your computer or home router. 

The latest version of the chip is only a prototype right now, so it will be a while before it gets installed and the Internet gets better.

However, IBM says it has already signed on a customer for an earlier version of the technology, a company called Semtech. Semtech makes a device that converts analog signals (like radio signals) to digital signals that can be piped across the Internet.

Equally interesting is that IBM says it will manufacturer the chip for the Semtech deal in the U.S. at its semiconductor fab in East Fishkill, N.Y.

That's of note because there's been speculation that IBM may be looking for a buyer for its semiconductor manufacturing unit. Breakthrough technology like this could either help the unit grow revenues, allowing IBM to keep it, or allow IBM to sell it for a higher price.

By: Julie Bort
Link: http://www.businessinsider.com/ibm-chip-makes-the-internet-faster-2014-2

Up Close and Personal With IBM PureApplication PaaS

The converged infrastructure value proposition, by now, is pretty evident to everyone in the industry. Whether that proposition can be realized, is highly dependent on your particular organization, and specific use case. 

Over the past several months, I have had an opportunity to be involved with a very high-profile pilot, with immovable, over-the-top deadlines.  In addition, the security requirements were downright oppressive, and necessitated a completely isolated, separate environment. Multi-tenancy was not an option. 

With all this in mind, a pre-built, converged infrastructure package became the obvious choice. Since the solution would be built upon a suite of IBM software, they pitched their new PureApplication system. My first reaction was to look at it as an obvious IBM competitor to the venerable vBlock. But I quickly dismissed that, as I learned more. 

The PureApplication platform is quite a bit more than a vBlock competitor. It leverages IBM’s services expertise to provide a giant catalog of pre-configured multi-tiered applications that have been essentially captured, and turned into what IBM calls a “pattern”. The simplest way I can think of to describe a pattern is like the application blueprint that Aaron Sweemer was talking about a few months back. The pattern consists of all tiers of an application, which are deployed and configured simultaneously, and on-demand.

As an example, if one needs a message broker app, there’s a pattern for it. After it is deployed (usually within 20-30 mins.), what’s sitting there is a DataPower appliance, web services, message broker, and database. It’s all configured, and ready to run. Once you load up your specific BAR files, and configure the specifics of how inbound connections and messages will be handled, you can patternize all that with script packages, so that next time you deploy, you’re ready to process messages in 20 minutes.  If you want to create your own patterns, there’s a pretty simple drag and drop interface for doing so. 

I know what you’re thinking. . . There are plenty of other ways to capture images, vApps, etc. to make application deployment fast. But what PureApp brings to the table is the (and I hate using this phrase) best-practices from IBM’s years of consulting and building these solutions for thousands of customers. There’s no ground-up installation of each tier, with the tedious hours of configuration, and the cost associated with those hours. That’s what you are paying for when you buy PureApp. 

Don’t have anyone in house with years of experience deploying SugarCRM, Business Intelligence, Message Broker, SAP, or BPM from the ground up? No problem. There are patterns for all of them. There are hundreds of patterns so far, and many more are in the pipeline from a growing list of global partners. 

The PureApplication platform uses IBM blades, IBM switching, and IBM V7000 storage. The hypervisor is VMware, and they even run vCenter. Problem is, you can’t access vCenter, or install any add-on features. They’ve written their own algorithms for HA, and some of the other things that you’d expect vCenter to handle. The reasoning for this, ostensibly, is so they can support other hypervisors in the future. 

For someone accustomed to running VMware and vCenter, it can be quite difficult to get your head around having NO access to the hosts, or vCenter to do any troubleshooting, monitoring, or configuration. But the IBM answer is, this is supposed to be a cloud in a box, and the underlying infrastructure is irrelevant. Still, going from a provider mentality, to an infrastructure consumer one, is a difficult transition, and one that I am still struggling with personally. 

The way licensing is handled on this system is, you can use all the licenses for Message Broker, DB2, Red Hat, and the other IBM software pieces that you can possibly consume with the box. It’s a smart way to implement licensing.  You’re never going to be able to run more licenses than you “pay for” with the finite resources included with each system. It’s extremely convenient for the end user, as there is no need to keep up with licensing for the patternized software. 

Access to the PureApp platform is via the PureApp console, or CLI. It’s a good interface, but it’s also definitely a 1.x interface. There is very extensive scripting support for adding to patterns, and individual virtual machines. There are also multi-tenancy capabilities by creating multiple “cloud groups” to carve up resources.  There are things that need to be improved, like refresh, and access to more in-depth monitoring of the system.  Having said that, even in the past six months, the improvements made have been quite significant.  IBM is obviously throwing incredible amounts of resources at this platform. Deploying patterns is quite easy, and there is an IBM Image Capture pattern that will hook into existing ESXi hosts to pull off VM’s to use in Pure, and prepare them for patternization.

Having used the platform for a while now, I like it more every day. A couple weeks ago, we were able to press a single button, and upgrade firmware on the switches, blades, ESXi, and the v7000 storage with no input from us. My biggest complaint so far is that I have no access to vCenter to install things like vShield, backup software, monitoring software, etc.. But again, it’s just getting used to a new paradigm that’s hard for me.  IBM does have a monitoring pattern that deploys Tivoli, which helps with monitoring, but it’s one more thing to learn and administer. That said, I do understand why they don’t want people looking into the guts on a true PaaS.

Overall, I can say that I am impressed with the amount of work that has gone into building the PureApplication platform, and am looking forward to the features they have in the pipeline. The support has been great so far as well, but I do hope the support organization can keep up with the exponential sales growth. I have a feeling there will be plenty more growth in 2014. 

By: Brandon Riley
Link: http://www.virtualinsanity.com/index.php/2014/02/10/up-close-and-personal-with-ibm-pureapplication-paas/

Server market realignment

The server market is in the midst of a radical realignment, the likes of which have not been seen since the shakeout of the 1980s that saw most of the minicomputer makers, including Prime Computer, Data General and Digital Equipment Corp., disappear, devastating the Boston high tech corridor. And while the writing has been on the wall for some time, this major industry shift promises to happen much faster than that one.

IBM System x General Manager Adalio Sanchez speaking at an IBM event in Beijing on January 16, 2014 to debut the company’s latest x86-based servers. Today IBM announced plans for Lenovo to acquire IBM’s x86 server business for $2.3 billion.

The first major shock came to the market last month, when IBM announced an agreement to sell its System x servers, x86 network switches and other x86-based products to Lenovo, continuing IBM’s transition into a software and services provider. While internal sources say that the sale, which includes the transfer of up to 6,700 IBM employees to the commodity system maker, will take several months to complete, this announcement definitely points to the future of x86 hardware.

Actually the commodization of x86 has been ongoing for several years and is well under way. It started with the invention of hyperscale by the big Web service companies including Yahoo, Google, Amazon, and Facebook. These companies buy huge quantities of standardized white box servers direct from Taiwan and China for their mega-data centers, run them hard in highly automated environments and, when something breaks, throw it away and replace it with a new box. But even before that the seeds of commodization were sown by the major traditional players themselves when they handed manufacturing of their servers over to the Taiwanese. Essentially they created their own replacements.

That arrangement worked for them as long as the hardware still required lots of attention, expensive built-in management software, and constant optimization fine tuning to handle the compute loads. But in the last decade three things have changed. First, Moore’s Law has driven compute power and network speed to the point that detailed optimization is no longer necessary for most compute loads. Second, the management software has moved to the virtualization layer. The result of these two trends is that increasingly the focus of IT organization attention is moving up the stack to software, and hardware is taken for granted. After 67 years, the techies are finally tiring of fiddling constantly with the hardware.

Third, increasing amounts of the compute load is moving steadily to the cloud. Companies that always had to buy extra compute to support peak loads now can move those applications into a hybrid cloud, size their hardware for the average load and burst the peaks to their public cloud partner. As those companies gain a comfort level with their public cloud service providers, they will start moving entire compute loads, particularly new Web-based applications such as big data analysis that have a strong affinity to the public cloud, entirely to those providers, in many cases by subscribing to SaaS services.

Trend toward standardization

 

The result of this is that the underlying hardware is becoming highly standardized, and the focus of computing is moving to software and services. Under the onslaught of hyperscale and cloud computing, the market for the traditional vendors is decreasing, a trend that will accelerate through this decade. And the market is shifting from piece-parts to converged systems as customers seek to simplify their supply chains and save money. As Wikibon CTO David Floyer points out, the more of the system that can be covered by a single SKU, the more customers can save. The hardware growth for both IBM and HP is clearly in their converged systems, and their differentiation increasingly comes from what they provide above the virtualization layer in middleware and applications. The expansion of virtualization from servers to the software-led data center will only drive this trend faster.

Open source hardware is beginning to appear in the market and can be expected to become commonplace over the next five years as the big Asian white box makers adopt them as the next step in driving cost out of the server manufacturing process.

The clear message for x86 server vendors is either to drive cost out of their hardware business and become commodity providers on the level of the Taiwanese while developing differentiation through higher level software running on top of those commoditized boxes or get out of the x86 hardware business entirely and source their servers from a commodity provider. IBM has clearly chosen the latter course with its sales of System x to Lenovo along with the creation of a close partnership with the Chinese commodity hardware manufacturer.

IBM’s strategy — partner with Lenovo

This is the right strategy for both companies. Since buying IBM’s PC manufacturing business a decade ago, Lenovo has proven itself as a quality commodity electronics maker, and in the process passing HP last year to become the number one PC vendor in worldwide sales. IBM, meanwhile, is a highly creative company with a huge R & D budget that is betting its future on leading edge areas including big data processing and analysis, business cloud services worldwide, and Watson.

The close partnership should leverage the very different strengths of both companies to create products that benefit from both, particularly in the IBM PureSystems line of integrated systems. Meanwhile Lenovo is likely to enter the hyperscale market once it has brought its manufacturing and marketing to bear on the IBM System x line. It also will certainly continue to sell its rebranded System x servers into the traditional business and governmental markets and can be expected to field its own x86-based converged system line probably in partnership with IBM. Since both companies will be profiting from the relationship, regardless of whose brand is on the box, they both will have strong business reasons to maintain a close partnership into the future.

IBM is in the market position it is in today in large part because of the visionary leadership of CEO Louis V. Gerstner Jr., IBM’s head for a decade through much of the 1990s and early 2000s. He foresaw the industry changes we are experiencing today, at least in their general form, and espoused the strategy of transforming IBM from a hardware giant to a software and services company. In retrospect this was prophetic, and while obviously nobody in 1993 could have anticipated the impact of cloud computing and in Gerstner’s time “services” mostly meant consulting, he moved IBM in a direction that puts it in a strong position today to capitalize on the burgeoning cloud services market. And his successors, Samuel J. Palmisano and Virginia M. Rometty, have continued to move IBM forward and make the hard decisions sometimes necessary for IBM’s transformation.

HP at the crossroads

But what about the other big x86 server vendors, who did not have the good fortune to have a visionary at their helm in the 1990s? HP in particular seems to have been searching for a path forward in recent years with its parade of short-lived CIOs. Carley Fiorina certainly had a strong vision, but unfortunately it proved to be the wrong one for the company.

After she left, HP suffered from a revolving door at the top. Michael Hurd was the only CEO to last long enough to create a vision for the company’s future, but he seemed mainly to see “more of the same”. Meg Whitman has been in charge for nearly two-and-half-years now and seems to have stabilized the company, but it is also suffering from market shrinkage. Its answer so far has been to bring forward some innovative hardware, but large parts of the company outside converged systems, Moonshot and storage seem to be going forward blindly, producing more of the same with no regard for the reefs ahead, and HPs financial results in recent quarters have shown the result.

HP needs to make up its mind, and fast. In its first two decades it was a highly creative, if sometimes chaotic, company producing leading edge products including some of the first servers and desktop printers. It really was the Apple Computer of its day. But it seems to have lost much of that, and today it buys more innovation than it produces in house, Moonshot not withstanding.

The problem is that while it has some very innovative products, they are all one-offs, and large parts of the company appear to be drifting. Decisions seem to be tactical rather than strategic. The PC group, for instance, is obviously floundering. At one time HP was almost as much a consumer company, with its desktop printers and PCs, as it was a business-to-business vendor. It has neglected that part of its business, which is a mistake.

HP seems to be moving in the direction of becoming a U.S.-based commodity hardware supplier. If that is what its leadership wants, then it should embrace that completely and start competing on price in all its markets while driving cost out its processes at all levels.  If it wants to return to its roots as a highly creative company then it should start building on products like Moonshot and revitalize its consumer and business market mindshare with new, creative electronic products that can create new markets. It cannot do both — no company can.

By: Bert Latamore

Link: http://siliconangle.com/blog/2014/02/04/server-market-realignment-cio-angle/

IBM successfully transmits text using graphene-based circuit

Big Blue confident it can use fragile material within smartphones and tablets.

IBM has successfully transmitted a text message using a circuit made out of graphene, as the firm shows the potential of carbon-based nanotechnology.

Graphene consists of a single layer of carbon atoms packed into a honeycomb structure. The low-cost material is an excellent conductor of electricity and heat, which meakes it ideal for use in smartphones and tablets as data can be transferred faster, save power and be more cost efficient.

The barrier to using graphene within integrated circuits is its fragility.  But IBM believes it has found a way to compensate for this weakness by using silicon as a backbone for circuits.

The firm created an RF receiver using three graphene transistors, four inductors, two capacitors, and two resistors. These components were packed into a 0.6 mm2 area and fabricated in a 200mm silicon production line.

Big Blue’s scientists were able to send and receive the digital text “I-B-M” over a 4.3 GHz signal with no distortion. The firm claims performance is 10,000 times better than previous efforts and is confident graphene can now be integrated into low-cost silicon technology.

The firm said applications could involve using graphene within smart sensors and RFID tags to send data signals at significant distances.

“This is the first time that someone has shown graphene devices and circuits to perform modern wireless communication functions comparable to silicon technology” said Supratik Guha, director of physical sciences at IBM Research.  

By: Khidr Suleman

Link: http://www.itpro.co.uk/mobile/21511/ibm-successfully-transmits-text-using-graphene-based-circuit

New IBM Kenexa talent suite taps Big Data to energize today's workforce

IBM today announced the software-as-service (SaaS)-based IBM Kenexa Talent Suite that allows Chief Human Resources Officers (CHROs) and C-Suite executives to gain actionable insights into the deluge of data shared every day by their workforce. As a result, organizations can now streamline, modernize and add precision to hiring practices, increase workforce productivity and connect employees in ways that impact business results.

Organizations around the world today are on a mission to identify and hire top talent. By hiring precisely the right employees and then arming them with powerful social tools, businesses can more effectively manage and develop their workforce and put them into the position to succeed.

With the IBM Kenexa Talent Suite, HR professionals can look at large volumes of employee data – such as work experience, social engagement, skills development and individual interests – to identify the qualities that make top performers successful. Organizations and teams can then use those models to pursue candidates through additional targeted social marketing on social recruiting sites, where job seekers matching the profile are automatically connected with opportunities matching their skills. 

Customers can accelerate the onboarding and the integration of new hires through IBM Connections capabilities. This helps employees share information and find the right experts to accelerate learning and increase productivity and engagement, while at the same time providing a way for leaders to more effectively manage their teams. Through analytics and reporting, line of business leaders can better understand emerging employee trends and more effectively manage each individual's career path in areas like skill attainment, performance appraisals, compensation, succession planning and more.

"We know people are the lifeblood of an organization, and business success on today's stage requires not just talent but social capabilities that can energize, empower and nurture each team member so they can reach their full potential," said Craig Hayman, General Manager, Industry Cloud Solutions, IBM. "By combining social, behavioral science and analytics in the cloud, we give businesses a clear path to empower their most valued asset – employees." 

Interested customers can complement the Suite with Watson Foundations, a comprehensive, integrated set of Big Data and Analytics capabilities that enable clients to find and capitalize on actionable insights. Watson Foundations provides the tools and capabilities to tap into relevant data – regardless of source or type – and apply a full range of analytics to gain fresh insights in real-time, securely across an enterprise. 

Using Watson Foundations customers will be able to conduct a deeper level of analysis on key workforce-related data, identify trends within the workforce, predict future trends and proactively take action. Executives can also look at the profiles and work performance of their top employees and determine the appropriate type of rewards needed to keep them engaged. 

According to an upcoming IBM C-Suite study that surveyed 342 CHROs representing 18 industries, many businesses are not taking full advantage of the insights delivered by workforce big data and analytics. The study found that just over half of organizations are using workforce analytics, with far fewer applying predictive analytics to optimize decision making and outcomes in areas such as sourcing and recruiting (7 percent), employee engagement and commitment (9 percent) and talent development (10 percent), retention (13 percent). 

The CHRO study also found that human resources executives are in the early stages of applying social approaches within the organization. Currently, 66 percent are regularly using social for their recruiting efforts, but only 41 percent are using it for learning 31 percent for knowledge sharing.

Today leading businesses such as AMC are benefiting from IBM talent management software. AMC, one of the world's premier entertainment companies, uses recruitment technologies from IBM to gain a deep understanding through data analytics of what it takes to succeed at the organization. AMC then uses that knowledge to attract candidates who are more likely to succeed once they're hired.   

"Harnessing the power of data gives us a better picture of what top talent looks like in our industry. IBM's talent management solutions allow us to use data in new ways so we can make better informed decisions that have a greater impact on our business," said Heather Jacox, Director, Diversity, Recruitment & Development at AMC.

The IBM Kenexa Talent Suite includes the following:

• Talent Acquisition: Includes recruitment, skill and behavioral science-based assessments and onboarding. These integrated functions are designed to provide a deep understanding of what the best talent looks like and then how to attract, hire and engage them.
• Talent Optimization: Includes performance management, succession planning and compensation planning to empower and get the most out of employees.
• Social Networking: Increases productivity with expertise identification and knowledge discovery – connecting employees and accelerating the time to productivity.

By: PR Newswire
Link: http://cloudcomputing.ulitzer.com/node/2942997

IBM announces revolutionary System x generation with modular design, 12 TB flash on memory bus

IBM today announced a complete redesign of its System x x86 server family, featuring up to 12.8 Tbytes of NAND flash directly on the memory bus of the server and a modular design that allows users to upgrade the server by simply replacing plug-in “Compute Books”. This sixth generation of System x has six core reference architectures including one for SAP HANA and is designed to facilitate the virtualization of ERP and other large, core enterprise-level applications for delivery through private and hybrid clouds. The scalable design can also reduce acquisition costs up to 28 percent in comparison to competitive Xeon x86 systems, IBM says.

The announcement includes a new System x3850 M4 BD storage server, a two-socket rack server supporting up to 14 flash and/or disk drives delivering up to 56 Tbytes of high-density storage. This server, which combines compute with storage, is specifically designed for large build-out architectures such as Hadoop big data installations, says Stuart McRae, IBM System x high-end business line manager.

It also includes the new IBM FlashSystem 840, providing double the bandwidth and performance – 1.1M IOPS – of its predecessor, the FlashSystem 820. It supports up to 48 Tbytes of usable capacity in a 2U unit with IBM Microlatency technology that cuts data access times to microseconds. Designed to support big data, it provides actionable insights from real-time data analytics faster than its predecessors. It also features a new management GUI and datacenter-optimized features such as hot-swap components and concurrent code load, enabling fast installation and easier management.

Virtualize your ERP for the cloud

“The System x6 is the first server family that’s been effectively designed from the ground up to incorporate flash storage,” McRae said. “Until now, flash storage has been kind of an add-on – you add on a PCI card to the server. This is integrating flash storage on the memory bus, the highest speed bus in the system, and making that available as a block storage device, that looks like any other block storage device to the application.”

By putting the flash on the memory bus, it becomes the fastest flash storage on the market. And these new systems can support a lot of it.

“It looks exactly like a DDR3 DIMM,” he said. “These systems are going to have 92 DIMM sockets in a four-way, so it can support up to 6 terabytes of system memory on a four-way, or 12 terabytes in an eight-way. That’s three times as much memory as is available in a standard eight-way server today.”

This has major implications both for big data analytics and for virtualization of very large enterprise applications such as ERP. “If you wanted to cache a five terabyte database in the server for analytics applications, you can configure that as a cache.” So for instance, in the SAP HANA appliance, a large amount of that space is used for RAM, allowing users to have a very large data set in HANA while still providing large amounts of flash for staging data. And by spreading a HANA or similar installation across several servers, it can support very large databases on the memory bus while providing resilience and redundancy in case of a hardware or power failure in any one server.

Before the x6 generation, users were memory-constrained in what they could virtualize. “If you had a four-way server only supporting one or two Tbytes of memory, it’s hard to virtualize a terabyte application,” McRae said. “Now they can do that on these new platforms.” This opens the way for virtualization of ERP systems, even Oracle Red Stacks, that today run on bare metal, allowing customers to realize the advantages of server virtualization and deliver services based on their ERP and other core systems to users via their private clouds.

“I want to move my large databases to a cloud model. I want to move my SAP HANA to a cloud model. I want to move my big ERP applications. I don’t want to have to re-architect it to a new architecture, I want to move it now, and this provides the infrastructure to do that,” MacRae said.

“Booking” your memory, flash, CPUs

The other part of the x6 revolution is the new parallel modular design that IBM calls “Compute Books”. Each server is made up of these plug-and-play modules, each with its own processor and memory. These plug into a backplane that provides power and IO.

This means that upgrading a server or replacing a failing unit is simply a matter of unplugging one or more modules and plugging in replacements. Then a simple restart implements the new hardware without requiring a forklift replacement and all the management that goes along with that. MacRae says IBM estimates that the core server will support at least the next three generations of processor and memory/flash technology.

“Once you’ve architected the server and put your big applications on it, two years from now, when you say ‘Scotty, I need more power,’ you just pull the Compute Books out and plug the latest, greatest ones in. It’s all transparent to the back-end IO.” That provides a great deal of investment protection across generations.

And while it does require a reboot of the upgraded server, “because it’s a virtualized environment, and now we’ve virtualized these large applications, you have no application downtime.”

Six reference architectures

As part of the announcement, IBM also announced six pre-architected versions that come with software installed: an SQL data Warehouse, a Hyper-V appliance running on Windows Server, an SAP HANA version, an SAP Business Suite version, a VMware vCloud, and finally a version running DB2 with BLU acceleration on Linux. The servers come with either SUSI or Red Hat Linux or Microsoft Server. While IBM does not have a reference architecture for it, Oracle has System x on its compatibility list, so users can also run an Oracle Red Stack on the new System x. And because of the higher end processors and the large amounts of memory and flash storage that the new generation supports, they can decrease the number of licenses they need, saving significant cost, particularly with Oracle. And System x also runs IBM Watson for users who want that in-house rather than using it from IBM’s cloud.

By: Bert Latamore

Link: http://siliconangle.com/blog/2014/01/16/ibm-announces-revolutionary-system-x-generation-with-modular-design-12-tb-flash-on-memory-bus/

IBM Introduces X6 Architecture, Optimizes x86-Based Servers for Cloud, Analytics

Industry-first integrated flash storage for high performance, modular design for long life cycles to help reduce costs, improves resiliency for increased uptime
New IBM FlashSystem 840 doubles performance, ideal for virtualization and cloud.

 

IBM today announced the sixth generation of its enterprise X-Architecture for System x and PureSystems servers, providing industry-leading improvements in the performance and economics of x86-based systems for analytics and cloud.

"Our enterprise X-Architecture for x86-based servers and solutions delivers high performance and the highest customer satisfaction in the industry, making us number one in high-end x86 systems," said Adalio Sanchez, general manager for IBM x-86 and PureSystems Solutions. "We continue to innovate and deliver leadership performance, reliability and investment protection for mission-critical workloads with X6."

Clients are rapidly adopting analytics for greater business insight and moving critical workloads like ERP, analytics and database to the cloud for increased efficiency and lower costs, and x86-based systems are the first choice for many. The X6 architecture represents IBM's continuing R&D investment and industry leadership in x86-based systems, and is specifically designed to provide new levels of performance and resiliency for enterprise applications. For memory-hungry applications, X6 delivers three times the scalable memory of current competitors' and IBM x86-based systems to support cloud and analytics. [1]

The X6 architecture is:

• Fast, with integrated eXFlash memory-channel storage -- an industry first, this DIMM-based storage provides up to 12.8 terabytes of ultrafast flash storage close to the processor, increasing application performance by providing the lowest system write latency available, essential for analytics applications. X6 can provide significantly lower latency for database operations, which can lower licensing costs and reduce storage costs by reducing or eliminating the need for external SAN/NAS storage units; [2]

• Agile, with a modular, scalable design that supports multiple generations of CPUs
  -- another industry first -- and can reduce acquisition costs, up to 28 percent in comparison to one competitive offering. [3] X6 provides stability and flexibility through forthcoming technology developments, allowing users to scale up now and upgrade efficiently in the future. Fast set-up and configuration patterns simplify deployment and life-cycle management;

• Resilient, with features that can help extend cloud delivery models to mission-critical applications. Memory and storage increase virtual machine capacity to allow SaaS delivery of applications. Autonomous self-healing CPU and memory systems maximize application uptime by proactively identifying potential failures and taking action to correct them. In addition, Upward Integration Modules can help reduce the cost and complexity of system administration by allowing operators to perform management tasks through virtualization tools. 

• Server models supported by this new architecture currently include the System x3850 X6 four-socket system, System x3950 X6 eight-socket system, and the IBM Flex System x880 scalable compute nodes. IBM also is introducing the System x3650 M4 BD storage server, a two-socket rack server supporting up to 14 drives delivering up to 56 terabytes of high-density storage -- the largest available in the industry. It provides 46 percent greater performance than previous comparable IBM System x servers and is ideally suited for distributed scale-out of big data workloads. [4] 

New Solutions for X6

Clients moving enterprise applications to cloud models and adopting analytics for quick business insights require integrated solutions for fast deployment, efficiency and performance. To help clients achieve these results, IBM is announcing new solutions for its X6 architecture for analytics, database and cloud deployment, including IBM System x Solution for DB2 with BLU Acceleration on X6 for accelerating analytics, IBM System x Solution for SAP HANA on X6 for analytics, and System x Solution for VMware vCloud Suite on X6 for infrastructure-as-a-service capabilities.

New Storage for cloud, analytics

IBM has announced the general availability of the new IBM FlashSystem 840. The new system provides nearly double the bandwidth and double the performance -- 1.1M IOPS -- of its predecessor, the FlashSystem 820 -- making it ideally suited for analytical databases, virtualization infrastructures, and public and private clouds. [5] Supporting up to 48 terabytes of usable capacity in a 2U unit, the all-Flash array also features IBM MicroLatency technology that significantly speeds data access times from milliseconds to microseconds (less than 135 microseconds) giving organizations faster actionable insights from real-time data analytics. In addition, a new management GUI - as well as datacenter-optimized features such as hot-swap components and concurrent code load - enable fast installation and easy management.

IBM also is introducing the FlashSystem Enterprise Performance Solution, which bundles the FlashSystem 840 and IBM System Storage SAN Volume Controller (SVC) technology. The solution includes a suite of advanced data management features ranging from Real-time Compression, snapshots, thin provisioning, VAAI, and application aware copies, to FlashCopy, and storage virtualization with IBM Easy Tier.

New SDE capabilities for cloud

IBM is strengthening its software defined environment (SDE) portfolio with the introduction of IBM Platform Resource Scheduler for private and hybrid IBM SmartCloud clients who want to accelerate time-to-results, improve infrastructure flexibility and reduce operating costs. IBM Platform Resource Scheduler provides a fully virtualized, open and programmable architecture that ensures enterprises are taking advantage of all available IT resources -- from application software licenses to available network bandwidth.

Integrated with OpenStack, this dynamic resource management tool provides a comprehensive set of intelligent, policy-driven scheduling features that automatically allocate the right resources to the right job, balances workload demand with infrastructure supply and ensures adherence to service level agreements, improving overall application performance and efficiency. The open and extensible architecture also allows enterprises to easily reconfigure and add customized policies to meet their specific sharing and scheduling needs.

Financing for cloud and analytics

IBM Global Financing has announced new financing offerings today to help clients quickly adopt new cloud and analytics solutions by helping reduce upfront costs and speed return on investment. Credit-qualified clients can obtain Fair Market Value leasing when acquiring X6 architecture solutions. Other offerings announced include 0-percent financing and deferred payments for 90 days when they acquire IBM PureSystems, SDE and Storage technology.

Also, IBM's new mobile financing application can help IBM Business Partners close more business with their clients, allowing credit-qualified clients to acquire financing to deploy solutions for analytics, mobile computing, social business and Smarter Planet technologies quickly.

IBM Systems and Technology Group offers a full range of offerings supporting public, private and hybrid cloud implementations that integrate with IBM's cloud software and services. This Systems portfolio includes IBM System x racks and BladeCenter, NeXtScale, PureFlex, Power Systems and System z servers, and IBM Storage solutions.



[1] Triple the memory capacity (Up to 6TB in 4S system; up to 12TB in 8S system) and support up to 24 DDR3 DIMMs per socket and up to 64GB LRDIMM density, based on published Intel specifications, compared with x86 competitors with offerings based on Intel's current processors. 

[2] 5-10 microseconds write latency for eXFlash DIMMs in preliminary testing vs. 15-19 microseconds latency for PCIe-based flash storage from Fusion IO, Micron, and Virident, and 65 microseconds latency for Intel S3500 and S3700 SSDs. (Pending final IBM performance testing.) 
Using internal eXFlash storage reduces or eliminates the need for external SAN/NAS storage. Less SAN hardware means fewer software licenses. 

[3] 28 percent acquisition cost savings based on pricing of x3850 X6 at announcement on 2/18 vs. current pricing of a comparable x86 based system that includes 2 x Intel Xeon E7-4820 (v1) processors, 1TB of memory (16GB RDIMMs) 3.6TB of HDD storage, and Dual Port 10GBe SFP+ controller. x3850 X6 includes 2 Compute Books, 2 x Intel Xeon E7 processors, 1TB of memory (16GB RDIMMs), 3.6TB of HDD storage, and Dual Port 10GBe SFP+ controller. 

[4] Compared to HP two-socket servers supporting a maximum of 48 TB storage with 12 x 3.5" drives, and Dell two-socket servers supporting a maximum of 51.2 TB storage with 12 x 3.5" and 2 x 2.5" drives. 
46-percent figure based on Intel Internal Test Report #1310, using SPECjbb*2013 benchmark, July 2013. 

[5] The performance data discussed herein is presented as derived under specific operating conditions by IBM. Actual results may vary.




By: IBM News Releases
Link: http://www-03.ibm.com/press/us/en/pressrelease/42796.wss

The 2014 Australian Open Tennis Championship: Powered by IBM Cloud

Every January, Tennis Australia welcomes hundreds of thousands of tennis fans to Melbourne Olympic Park for the Australian Open Grand Slam. Millions more around the world arrive by australiaopen.com to catch the latest scores, match statistics, player insights and breaking news. All of this is powered by IBM data technologies.

Once again, IBM has partnered with Tennis Australia to deliver innovative IT solutions and a seamless online experience using its cutting edge technologies like cloud computing, mobility, social media sentiment and big data analytics. The partnership between IBM and Tennis Australia has been around for 21 consecutive years.

This year, the event will be held Jan. 13-26. In these two weeks, demand for information from Australian Open fans will be at its highest. During the tournament, Tennis Australia’s IT infrastructure must expand drastically to meet audience demand and deliver a unique and satisfying experience. How does Tennis Australia scale its IT infrastructure to accommodate the flood of data demand that comes from millions of fans at once?

Tennis Australia needed a highly scalable IT solution that would allow them to quickly and easily allocate a huge amount of additional resources. They needed a plan to cater to a short but extreme high-traffic period, in a timely and cost-effective way.

The solution was for IBM to set up a private cloud that could be shared by other like-minded organizations with similar demand spikes at different times of the year. Those organizations include the US Open, Roland Garros, Wimbledon, the US Open Golf tournament, the Masters and the Tony Awards. But such a solution doesn’t just apply to sports and entertainment. Any large companies considering sharing IT resources between internal divisions, departments or projects would benefit from such an arrangement.

Some of the technology solutions that IBM will provide to the tournament are: the official Australian Open tournament website, IBM ReturnServe, IBM SlamTracker, integrated end-to-end scoring system, Speed Serve System, Australian Open intranet, closed-circuit television (CCTV) feeds and the match information display located in Garden Square and Grand Slam oval.

The outcomes have been extremely positive. Tennis Australia:

• Can easily and affordably adjust their IT resources according to fluctuations in traffic and user demand, year-round.

• Is able to now free up operational budget for new investment thanks to leveraging virtualization, energy efficiency, standardization and automation.

• Can rely on an improved disaster recovery solution that locates back-up systems in separate facilities.

• Has grown its users by 45 percent since 2008, while cost per user has decreased by 35 percent.

• Has increased page views by 42 percent, while cost per page view has decreased by 34 percent.

• Has increase accuracy of demand predictions for the hosting infrastructure through additional input, including unstructured data from Twitter commentary.



By: Carlos Felipe Franca da Fonseca
Link: http://thoughtsoncloud.com/index.php/2014/01/video-how-ibm-cloud-is-powering-the-2014-australian-open-tennis-championship/

IBM rallies tennis fans with innovative technology at the Australian Open 2014

Tennis Australia and IBM use new mobile apps, enhanced data analytics and cloud computing to deliver the most engaging event yet for fans.

 

IBM is bringing tennis fans even closer to the Australian Open 2014, with new ways to access and interact with the latest tournament news and courtside action from the Grand Slam as it happens. 

Through the use of innovative and interactive technology, IBM will provide fans with a richer fan experience for the international tennis tournament. The official mobile apps for the Australian Open deliver easy functionality with real-time scores and schedules, comprehensive match and player analysis and Twitter feeds. New this year is the free iPad app, which is the ultimate digital destination, enabling fans to follow their favourite players, tweet messages of support from a player's profile and track player popularity online using IBM social media analytics.

“We are seeing increased demand from fans around the world for more access and real-time event content. Over the two weeks of the Australian Open last year, more than 15.5 million unique users connected with the tournament online, and almost half of the website views were from mobile devices.  Each year we aim to enhance the event experience and this year we have worked with our technology partner, IBM, to develop a new iPad app, improve the website and smartphone apps, as well as advance the technology infrastructure behind the scenes,” said Samir Mahir, CIO, Tennis Australia. 

Today’s tennis fans also seek a deeper experience that goes beyond live scores and updates. Insights and visualisations, provided by IBM SlamTracker on the Australian Open website, analyse more than eight years of Grand Slam data to identify patterns in player style. New features to SlamTracker for 2014 include streamlined analysis of more fan friendly key turning points in a match, such as aces and winning shots, and a social media sentiment feed that measures the percentage of positive tweets. 

“During the Australian Open each year we transform almost overnight from a medium size business to a global enterprise that must service millions of fans, players, media and officials from around the globe,” continued Samir Mahir. “Using IBM predictive analytics and cloud computing technologies ensures we can meet this demand uninterrupted.”

IBM predictive cloud provisioning analyses multiple data sources in real-time, such as tournament schedule, player popularity, historical data and social media conversations, to predict and automatically allocate the computing power required by the Australian Open website.

New this year, tennis fans can also now experience what it is like to face a serve from one of the world’s top players with the new IBM ReturnServe. To create this experience for fans IBM, in partnership with Tennis Australia, analyses real-time data from every point of the game at the Rod Laver Arena during the tournament. Displayed in a graphically animated environment and hosted on IBM SoftLayer, this live serve data lets fans attempt to return each serve at home using their computer or at select onsite locations using a virtual reality headset.

“Our 21-year history with the Australian Open has been about making the event and the sport of tennis more engaging and enjoyable.  This latest focus on the omni-channel experience is no exception. With new digital technologies like the iPad app and virtual reality IBM ReturnServe, this event is a leading example of how cloud computing, data analytics, social and mobile technologies can help organisations better connect with audiences,” said Glen Thomas, Vice President of Marketing and Communications, IBM Australia and New Zealand.

Australian Open 2014

IBM and Tennis Australia

Since IBM became Tennis Australia’s technology partner in 1993, the partnership has delivered innovative technology solutions, enhancing the Australian Open experience for players, coaches and fans around the world. Key IBM solutions at the 2014 tournament include:

• IBM SlamTracker - statistics and visualisation platform using predictive analytics and historical data to identify and broadcast key performance indicators, providing comparisons of players’ careers and live performances. This analysis includes key performance indicators — what players need to do to succeed in a match — known as 'Keys to the Match'. During the match, each player's performance is measured against their keys and updated in real time on ausopen.com, providing fans with a deeper level of insight as the match unfolds.

• Ausopen.com website - IBM designs, develops and hosts the official website on an IBM private cloud. Fans can follow live scores from every court, review highlights of the day’s play, listen to live tournament radio and read blogs.

• New in 2014 – IBM ReturnServe - Tennis fans can now experience what it’s like to face the world’s top players with the ReturnServe website and the ReturnServe virtual reality experience at selected locations in Sydney and Melbourne. Based on live data analysed by IBM technology at Rod Laver Arena, fans can face each serve at home, using their computer or at the virtual reality locations using a headset. The aim of the game is to return serves at the same speed as the world’s best. IBM ReturnServe is hosted on IBM SoftLayer, a robust and scalable platform.

• IBM’s integrated end-to-end scoring system - IBM technology drives the live scores that are broadcast around the world, along with other real-time data such as match statistics and the radar speed of serves.

• Mobile - the official Australian Open iPhone app, Android app and mobile website are built and powered by IBM, providing live scores, radio, Twitter feeds, schedules and draws. The mobiles apps were downloaded more than 1 million times during the 2013 tournament.

• New in 2014 - iPad app - the app allows fans to follow their favourite players and show their support through social media – as well as track how popular they are, thanks to IBM’s Social Sentiment analytics solution.

• IBM predictive cloud provisioning - IBM technology automatically assigns the computing power required by the Australian Open website based on real-time analytics of the tournament schedule, player popularity, historical data and social media conversations.

• Australian Open Social Leaderboard- updated every few minutes, this website feature tracks the positive and negative sentiment for players on social media. Fans can help their favourite players move up the ranks by ‘liking’ or tweeting favourably about a player.

• Scoring system - Vital scoring information is displayed via IBM’s integrated end-to-end scoring system, which makes live scores available around Melbourne Park and the world as they happen.

• Speed serve and radar - Courtside radar captures exact serve speeds, which are immediately transmitted to the IBM scoring database and displayed courtside.

• Chair Umpire Solution (CHUMP) and match statistics - Each chair umpire uses a mobile device to directly capture each point, game and set from the court. Courtside statisticians record vital information and the information is then sent in real time to the IBM Scoring System.

By: IBM Press Releases
Link: http://www-03.ibm.com/press/au/en/pressrelease/42932.wss