Computer Industry Shapes Direction of Broadcast Media Storage

The future of storage and how it's used is being determined today by the IT industry, where most of the components used in professional production and distribution systems and networks come from.

As the broadcast industry moves to file-based systems, storage is a key component in how facilities implement video servers into their workflow. Manufacturers have two options: use off-the-shelf IT solutions and tailor them for broadcast, or design custom storage systems into their products.

In the past, the highly specialized nature of broadcast had precluded the use of off-the-shelf technology — mainly due to performance reasons. However, with the advances we see today in the IT industry, some manufacturers have qualified specific high-performance computer storage systems for use in broadcast. The advantage here is in quicker time-to-market with new technology, generally lower costs for performance and more flexible solutions.

When we look at the storage landscape it's important to focus on two major elements—the physical storage media (e.g., internal data processing technology and form factor) and the workflow that it will ultimately be deployed in.

The Shape of Space

The IT industry designs storage for many markets, with two major customers, the PC market and the Enterprise market for mission critical type applications, driving most of the revenue. In many cases the design guidelines for these two markets are very different. The PC market is primarily focused on price while the Enterprise market demands high performance and reliability. Recently, however, new technology advancements have helped to create a new category between these two —the "Business Critical" market.

Within the Enterprise drive market there are several major trends that are driving the use of certain types of physical storage:

1. The Enterprise market is currently moving from Fibre Channel (FC) drives to serial attached SCSI (SAS) drives just as the PC market recently moved from parallel connections to serial SATA drives. Driving this migration is the fact that aside from the obvious cabling advantages of serial attach, serial clock speeds can be higher than parallel connections and SAS provides a direct point-to-point bus connection vs. a shared bus connection with FC SCSI drives resulting in better overall performance. This means stored material can be accessed much faster. For editors and program distributors, this results in better productivity.

   The performance of a drive is determined by its clock speed and its rotational speed (today Thomson uses 15,000 rpm SAS drives in its Grass Valley K2 Media Server). This determines how fast you can process and move data on and off the drive. In a video environment, the faster the storage system the better, but more importantly, it must be deterministic and consistent. In the IT industry, if you have a few millisecond pause in data transfers, nobody will notice, but in video, it can mean black frames.

   Video servers are designed with appropriate buffering to avoid these delays up to a certain point, which is why your storage system must be deterministic. An area that can cause these delays is disk failures in a RAID protected system — both FC and SAS drives provide good performance and deterministic behavior whereas SATA drives are not as predictable.

2. The latest version of SATA drives — SATA 3Gb/s, running at 7,500 rpm, have made a lot of improvements over first generation SATA 1.5Gb/s drives. In addition to clock speeds doubling to 3 Gb/s which doubled the transfer rate to 300 MB/s, features such as hot-swap capability, improved MTBF (mean-time between failure) and Native Command Queuing (which enables the drives to internally optimize how commands are executed for better performance) opens new markets for SATA drives. While SATA drives continue to get better, they still lag behind that of Enterprise SAS drives for high-performance, high reliable mission critical markets.

   The "Business Critical" Market

   These advancements have created a new class of SATA drives, which can be half the cost of SAS drives and becomes a practical consideration for use in lower performance video applications. Seagate, which supplies a large number of drives to the broadcast and video production industry, calls them "Business Critical" drives; indicating a class of drives between Enterprise SAS drives and the low cost, lower performance and less reliable PC drives.

   We consider the broadcast market to be a "Mission Critical" market. While a disk problem will not result in a fatality, it can result in lost revenue with make-goods. Most broadcasters demand that their server system be as reliable as possible, which has driven the almost exclusive use of Enterprise Drives in professional video servers. However, new advances in these Business Critical devices make them an ideal choice to support some types of video production and less demanding playout applications. These SATA drives will also be available with a SAS interface. This allows you to pick a storage system and populate it with either drive, based on your application and budget. These should begin appearing in the community later this year.

3. The move from 3.5-inch drives to 2.5-inch Enterprise drives will result in more compact storage systems that offer more gigabytes (GB) per square inch. This means smaller libraries and servers that hold much more material than ever before. Broadcast stations and mobile production vehicles with limited space will appreciate these new 2.5-inch drives as they start to appear later this year and into 2009.

   Coinciding with this smaller form factor, the capacity of storage drives continues to increase at an almost unbelievable rate. Today in the PC market we see low-performance 1 TB drives, with 1.5 TB drives coming shortly. The high performance Enterprise market is currently at 300 GB drives with 450 GB coming later this year and 600 GB drives following in the next 12-18 months. Perpendicular recording technology is allowing the disk industry to pack more bits per square inch, resulting in the need for less physical platters (smaller space) while enjoying higher performance.

4. A small but growing segment of the storage industry is also moving away from spinning discs to solid-state compact flash memory. The price per GB is still prohibitive to make this practical for most video applications at the moment, but future economies of scale will make solid-state storage an option for some segments in the production industry. Production trucks or harsh environments that need extreme robustness over massive storage will embrace this because solid-state is not impacted by mechanical vibrations such as road bumps or earthquakes.

   We're starting to see solid-state drives in consumer PCs (e.g., the new Apple Air) and that trend will continue. The attraction of no moving parts, instant access and fast read time (up to 10x faster than disks) is just too compelling to ignore. However, solid-state is not without a downside — this technology is optimal for reading but has limited write capability. Memory cells have limited number of writes. To counter this solid state disks are designed with sophisticated technology that avoids writing data to the same cells and includes additional memory that can be used as storage areas approach their limits. Spinning disks will always offer an advantage in terms of price/GB, but solid state will continue to gain market share over the next few years.

   It's become clear that different applications will gravitate to the drive technology that best suits its needs, based mainly on performance first and price second. SAS drives will certainly be considered for online editing and fast access, but the less costly "Business Critical" drives are sure to make significant inroads within the production community. Their performance is certainly good enough for a lot of applications where video I/O is important but less critical.

Improving Workflow Efficiency

The overall concept of servers has moved facilities from a baseband video infrastructure to a file-base infrastructure, bringing with it a long list of benefits that this type of IT-centric architecture affords. Broadcasters and production studios are seeing more and more success with cost savings and workflow efficiencies in migrating this way.

The whole idea of a tapeless facility goes back to implementing a workflow that gets out of real-time 270 Mb/s (or 1.5Gb for HD) video and into a data file that can be as low as 8 Mb/s for transfers many times faster (15-25 Mb/s is typical for SD and 50 Mb/s for HD). The most benefit is realized by making the conversion (encoding) as early in the process as possible, preferably during the ingest process.

In addition, with file-based workflows you have the option of working with a low-resolution, browse-level version of the file (1 Mb/s or less) for QA purposes, editing, quick review or archiving. This allows you to cost-effectively develop internal networks where hundreds of journalists and producers can access the same file at the same time, while keeping bandwidth requirements low.

Tiered Storage Strategy

Another option for tapeless workflows is to implement a tiered storage strategy, in which you have three types of storage; on-line, near-line and off-line.

• On-line = Enterprise drives, highest performance, reliability and highest cost. Usually configured in a SAN system.
• Near-line = SATA drives, moderate performance, good reliability and lower cost. Usually configured in a NAS system.
• Off-line = tape archive; lowest performance and lowest cost. Storage robotic systems can be small as a desktop or as large as a bedroom.

While on-line and off-line storage is a common strategy today, the addition of near-line storage offers significant cost savings and performance improvements. For example, in a news production environment, you might want to store footage on-line for one week, move it to near-line for 30 days and then to off-line. This reduces the amount of on-line storage without the large performance penalty of tape.

Storage Infrastructure Trends

The storage infrastructure is what ties everything together. Today in the Enterprise market we have FC and GigE connections. FC has always been the performance leader but is costly to implement.

Gigabit technology has been predominate in the IT industry and we are seeing its performance advance to where it is a good alternative to FC while coming at a lower cost (with cheaper switches and cabling). Today FC has moved from 2 Gb/s to 4 Gb/s performance. Ethernet is predominately 1 Gb/s, but we're seeing cost-effective 10 Gb/s switches (actually a few 10 Gb ports on a 1 Gb switch) being implemented. New technologies such as iSCSI (SCSI commands over Ethernet) and TOE cards (TCP/IP engines required for off-loading the system CPU) make GigE a good option for high performance, deterministic video systems.

For the most common server implementations 1 Gb/s performance is certainly good enough, but when you want to move a massive amount of data in and out of a server, the more bandwidth you have available the better. This is where 10 Gb/s Ethernet becomes ideal. Often a high performance ftp network will be mostly 1 Gb/s with a 10 Gb/s backbone. This allows most devices to talk to the network via 1 Gb/s but gives some devices the option of 10 Gb. For example, if you need to move data to a very fast archive system with multiple tape drives in the 120 MB/s range, then you can easily max out a 1 Gb/s connection.

Embracing File Wrappers

Another important aspect to achieving workflow improvements is the constant advancement of file wrappers and how video data is stored within such protocols as the Media eXchange Format (MXF), general Exchange Format (GXF), QuickTime and others like it. The attempt to achieve a common file format that is interoperable across disparate manufacturers' platforms is gaining good success, but at present we're still seeing some ambiguity in how the standards are implemented. File structure, with regard to metadata, can be implemented differently. The industry is aggressively resolving these issues. We feel MFX is the best option for a truly open interoperability standard in our industry. This will only get better with time, as there's a real commitment among manufacturers and the industry in general to achieve this goal and help streamline production processes that at present can be counter-productive.

QuickTime (QT) continues to add features and capabilities (now with QT V7) that make it a popular option in the production industry with almost all NLEs supporting it. This has been invaluable since it's become clear that the more metadata you can share, the more powerful, useful and productive your application and workflow will be.

Compression Still Matters

Finally, although the price of storage continues to decline, it doesn't mean that the industry will some day soon be working uncompressed. In fact, the trend is just the opposite. Practical workflows will continue to rely on the use of compression in order to move files around a facility quickly and unfettered by network bandwidth. Newly emerging compression codecs like H.264 and AVC HD are improving and providing higher quality while using lower bit rates.

For example in the 1990s, 50 Mb/s MJPEG was the standard for SD broadcasts. Today 50 Mb/s MPEG II LGOP is the standard for HD, with six times the amount of data. Technologies such as H.262 and AVCHD can cut this by 50 percent or more.

This has resulted in infrastructures that are less expensive to deploy while enabling users to move these smaller files around faster. It also allows producers to distribute content outside the facility using less bandwidth, whether the route be via satellite or fiber-optic lines, which have a high cost for bandwidth. That's where broadcasters want to go.

Conclusion

Going forward, as storage demands increase — which is a certainty given the need to support multiple channels of HD content sent to a variety of distribution platforms — IT-centric technologies will continue to provide the solutions broadcasters require. Some applications are still being developed, so no one's sure exactly what technologies will be adopted most. What is clear is that many vendors serving the broadcast industry have recognized the efficiencies to be found in off-the-shelf IT solutions. While some still cling to proprietary architectures, it's the most open approaches that will provide the fastest return on investment and ultimately be most successful.

Roger Crooks is Product Marketing Manager for Servers and Digital News Production Products at Grass Valley.