Difference between revisions of "Storage Technology and TouchDesigner"

From TouchDesigner Documentation
Jump to: navigation, search
Line 99: Line 99:
 
6 SATA-III ports (interface) with SATA Cable
 
6 SATA-III ports (interface) with SATA Cable
  
[[File:Sata.png|frame|right|256px]]
+
[[File:Sata.png|256px]]
  
 
SATA-III is the latest version of SATA and has already been around for over a decade.  SATA-III is the most widely supported bus interface found on today’s motherboards.
 
SATA-III is the latest version of SATA and has already been around for over a decade.  SATA-III is the most widely supported bus interface found on today’s motherboards.

Revision as of 21:48, 19 March 2019

Overview[edit]

Understanding storage technology can be daunting because there are a wide range of compatible and incompatible systems, some software, some hardware, some combinations of both. There are hardware interfaces, software protocols, cables and connectors, form factors, and media types. Some of the systems overlap, others are phasing out and yet others are catching on.

Today’s storage landscape is changing quickly, and is likely an area where errors will are made when assembling choices for a media server.

The following chart outlines the key technologies in storage for building a modern server. The scope is limited to “inside the box” so we will not cover network based NAS systems that stream data over a network or external drives that connect via USB 3 or Thunderbolt.

From a high level, its important to have familiarity with the following technologies.

Storage Technology Chart[edit]

Technology Type Intro Date Prevalence Life Cycle
HHD Magnetic Storage Media 1956! Ubiquitous Standard
NAND Storage Media 1984 Ubiquitous Standard
SSD Storage Media 1991 Ubiquitous Standard
SCSI Protocol / Interface / Bus 1986 Antique Discontinued
RAID Technology 1987 Ubiquitous Standard
SATA-I Interface / Bus 2000 Antique Discontinued
SATA-II Interface / Bus 2004 Rare Discontinued
AHCI Protocol 2004 Ubiquitous Twilight
SATA-III Interface / Bus 2008 Ubiquitous Twilight
PCI Express 3.0 Bus 2010 Ubiquitous Standard
SATA Express Interface / Protocol 2013 Rare Fail
SAS Protocol / Interface Enterprise / Pro Standard
mSATA Interface Mobile Only Standard
M.2 Interface Ubiquitous New Standard
NVMe Protocol Growing New Standard
U.2 Interface Rare New
PCI Express 4.0 Bus Rare New

Storage Media[edit]

Storage media refers to the place where data is physically stored. When considering mass storage devices for a TouchDesigner server there are two broad choices. In general, server data can be stored on magnetic spinning disks or it can be stored on nonvolatile solid-state flash memory which has the ability to store data when not powered.

A magnetic spinning disk is know as a hard-drive. Non volatile solid state flash memory is know as NAND or v-NAND.

Storage Media Form Factors[edit]

Today magnetic disks are generally referred to as hard-drives or HHD. Form factors are generally specified in inches, and for most TouchDesigner applications there will be only 3.5 inch drives for consideration.

SSD media comes in an array of formats that work on a wide range of interfaces like USB, SATA, PCI boards and M.2 slots etc.  SSD drives also work with both the SATA protocol and NVMe protocol.

HHD vs SSD[edit]

HHD[edit]

HHD’s work with spinning platters and read-write heads that can move to any location at tremendous speeds. The speed at which the platters rotate is measured in RPMs and larger numbers usually mean faster performance.  Because HHD’s are partially a physical machine there are limits to the maximum possible speeds and as such hard drive technology has plateaued in the context of how quickly data can be retrieved or stored.

Capacities of HHD drives have grown well over 10TB and therefore HHD storage media is still very competitive for offline storage in video servers. Offline storage can contain large video files, rendered raw footage etc, cloud data images etc.

HHD drives can be found with interfaces for both SATA and SAS interfaces. Recommended speeds start at 7200 RPMs with suggested cache sizes above 64 MBs.

Modern HHD drives are generally fast enough to write and store H.264 or H.265 live encoded video so if you are building a video recording system, a large capacity HHD system configured in a variety of possible RAID setups might be the best option.

SSD[edit]

SSD storage media is the standard for modern high performance storage. An SSD (solid-state drive) is a configuration of v-NAND, form factor, flash controller, interface and protocol. The architectural configuration of the SSD controller is optimized to deliver high read and write performance for both sequential and random data requests. SSDs are sometimes referred to as flash drives or solid-state disks.

Because SSD drives are just arrays of fast memory, the interfaces and protocols that were developed specifically for HHD technology, namely SATA and SAS protocols are now antiquated for SSD architectures.

You can still find SSD drives configured with SATA or SAS interfaces but these are quickly becoming a thing of the past. The following section covers a range of interfaces and protocols that are available today.

Interfaces, Protocols, Controllers and Busses[edit]

Before the turn of the millenium, there were a range of protocols, interfaces and buses that permitted computers to connect with peripherals like printers, disk drives and scanners. One of the most widely integrated technologies was called SCSI or Small Computer System Interface. There were others like AT Attachment (ATA) and subsequently Parallel ATA or (PATA).

By today’s standards the SCSI and ATA protocols and interfaces are outdated in large part due to their bandwidth, electronics and software architecture, prohibiting the massively parallel access to data which is now expected with expansion devices like graphics cards and SSDs.

The following section describes the interfaces and protocols that have been developed over the last 18+ years that were designed to overcome the limitations of protocols like SCSI, looking into recent trends to distill how each of the technologies might benefit a TouchDesigner powered server architecture.

SATA-I, II, III[edit]

The most successful and now widely available technology that replaced the SCSI and PATA protocols is SATA or Serial ATA. SATA is actually the evolution of the PATA interface standard. Serial ATA industry compatibility specifications originate from the Serial ATA International Organization (SATA-IO).

SATA is a computer bus interface that connects a compatible hard drive to a host bus adaptor. The host bus adaptor is usually found as a chip on the motherboard that will manage connected devices, permitting communication with the other subsystems including the CPU.

A SATA III bus has a maximum throughput of 6 Gb/s (Gigabits) per hard drive. There are 8 bits in a byte so there is a theoretical maximum bandwidth of 600 MBs, which in the real world ends up closer to 550 MBs maximum per drive.  

6 SATA-III ports (interface) with SATA Cable

Sata.png

SATA-III is the latest version of SATA and has already been around for over a decade.  SATA-III is the most widely supported bus interface found on today’s motherboards.

The SATA-III interface is supported on a wide range of both SSD and HHD storage media.

SSD with SATA-III Interface File:Images/image7.png


AHCI Protocol ( Advanced Host Controller Interface )[edit]

AHCI is a standard and protocol. It was defined by Intel and was developed to improve how the computer operated with the SATA host controllers.

When the SATA controller was introduced, there were various proprietary protocols that required specific drivers and suffered from a range of inadequacies. AHCI provided a single standard that is now widely adopted by motherboard manufacturers, BIOS developers and operating systems.

Most SATA controllers can run in either “legacy mode” (also known as “IDE”) or AHCI mode. When building a TouchDesigner server where the operating system will be installed on a SATA drive, it’s important to first ensure that the BIOS has that SATA controller set to AHCI mode.

When a SATA host controller is in AHCI mode, the controller is able to support a larger number of drive ports, supporting up to 32 devices over the legacy mode which is likely 4 devices.

SATA + AHCI has been the industry standard for personal computer hard drive solutions for many years.  When using SATA drives not configured in RAID ( RAID is explained briefly in next section ), AHCI should be used.

File:Images/image9.png[edit]

RAID - Redundant Array of Independent Disks[edit]

RAID technology has been around and evolving for many decades.  All three storage communication protocols, SATA, SAS and NVMe support RAID configurations. RAID is very big topic and if you are unfamiliar with this technology please refer to the internet.  Wikipedia very adequate coverage here…

https://en.wikipedia.org/wiki/RAID

SAS - Serial Attached SCSI[edit]

Serial Attached SCSI is an alternative protocol for moving data from storage devices to the computer.  SAS is designed and maintained by the International Committee for Information Technology Standard (INCITS).  

Unlike SATA, SAS controllers are generally installed as PCIe SAS expansion boards although embedded SAS controllers can be found on enterprise server motherboards as well.

 

The most widely available SAS controllers today are SAS-3 which supports 12 Gbit/s and have been available since 2013.  However the SAS-4 standard was completed in 2017 and is known as 24G, sporting a much faster 22.5 Gbit/s bus.

Because SAS drives are faster than SATA and are generally marketed as enterprise and professional server storage solutions, they are more well suited to RAID configurations either as RAID 0 for greater speed and capacity or other configurations to support data redundancy.

Typical SAS-3 12 Gbit/s Controller PCIe Card

File:Images/image4.png

NVM Express (NVMe)[edit]

As indicated above, the current SATA-III bus is limited to a maximum of 600 MBytes/s and SAS controllers will limit each drive to a maximum of around 1.2 GBytes/s.  Both technologies were developed before the introduction of SSD drives so there were particular limitations in the architecture that didn’t matter with only HHD spinning drives.

As well, both AHCI / SATA and SAS protocols have limited architectures for requesting and storing data onto any hard-drive whether it’s an HHD or SSD. Flash memory doesn’t have the same physical limitations as spinning platters with read write heads, so a new protocol was designed specifically for SSD drives.

A new specification was introduced in 2013 called the Non-Volatile Memory Host Controller Interface or NVM Express.  The express designation is an indication of where it runs.  Instead of using the antiquated SATA controller, the much faster PCI Express lane would be leveraged to dramatically increase the bandwidth at which data could flow from the controller to the CPU.

As well the new NVMe specification included new ways to communicate with the arrays of flash memory increasing limits on simultaneous instructions of AHCI’s maximum of 32 to a theoretical maximum have 64,000.  

Over the last few years the advancement of storage speeds running on NVMe technology has been extremely impressive.  

PCI Express for Storage[edit]

A 2x PCI 3.0 lane supports 2 GB/s substantially outdoing today’s SATA III 600 MB/s interface. A 2x PCI 4.0 lane will support 4 GB/s which is incredibly promising technology due out this year.

NVMe drives will typically take up 2 to 4 lanes for each drive. When you are looking at specs you should be looking and the PCIe lane version and number of lanes to calculate what you need supported by your CPU.

PCI V

Introduced

Transfer

rate[i]

Throughput[i]

×1

×2

×4

×8

×16

1.0

2003

2.5 GT/s

250 MB/s

0.50 GB/s

1.0 GB/s

2.0 GB/s

4.0 GB/s

2.0

2007

5.0 GT/s

500 MB/s

1.0 GB/s

2.0 GB/s

4.0 GB/s

8.0 GB/s

3.0

2010

8.0 GT/s

985 MB/s

1.97 GB/s

3.94 GB/s

7.88 GB/s

15.8 GB/s

4.0

2017

16.0 GT/s

1969 MB/s

3.94 GB/s

7.88 GB/s

15.75 GB/s

31.5 GB/s

NVMe Hardware Interfaces and Memory Form Factors[edit]

If you want an SSD drive you will have to decide on an interface and form factor.  This section will introduce you to the options.

mSATA[edit]

mSATA does not support NVMe however its worth introducing this little drive because its available for small form factor server ideas. The mSATA standard was developed to support thin notebooks and tablets.  mSATA technology is limited to very small boards which don’t have room for many SSD V-NAND chips so the upper capacity for a single drive is very limited. Like SATA III, mSATA drives are also limited by the same 600 MB/s bandwidth limit.

File:Images/image12.png

File:Images/image6.png

M.2 [edit]

M.2 was designed to overcome the capacity and speed limitations of mSATA. In many cases it has already become the desired replacement for the mSATA standard because its thinner and can sustain more v-NAND chips. M.2 devices can be connected to SATA ports or directly the to PCI Express 3.0 lanes, completely circumventing the SATA bandwidth limited SATA protocols.

In cases where the M.2 slot is connect the SATA controller it will have the same limitations. When the M.2 slot is connected to the PCI bus it will have access to much higher speeds and will be using the NVMe standard.

If you find a cheap M.2 you should be careful it isnt an AHCI / SATA only SSD if you want a high speed NVMe based M.2 drive.  For example the following M.2 is only SATA and therefore limited to SATA speeds.  If you look closely it says M.2 SATA on the chip.

File:Images/image3.png

Meanwhile this slightly more expensive model supports NVMe and much higher data rates.

File:Images/image8.png

M.2 Modules can come in longer lengths like 42, 60 and 80mm.  Longer modules fit more NAND chips and therefore can support greater capacities.

The M.2 slot is just a format and a slot time with a hardware connection to the motherboard.  It simply defines a connector and a length for device manufacturers to comply with. Motherboards have M.2 slots that work with a variety of technologies and protocols and therefore you will need to distinguish between them.

U.2[edit]

Like M.2, the U.2 interface has been developed to physically connect SSD drives directly the PCI Express bus. Currently each u.2 interface can work directly on a PCI 3.0 x4 bus for a theoretical maximum bandwidth of 4 GB/s per drive. The advantage of the U.2 Controller is that the port format is based on a wire connection and looks similar to the SAS Controller connector.

This means you can connect a 2.5” SSD using a cable and place the SSD drive into hot swappable drive bays for easy access.

http://highpoint-tech.com/USA_new/series-ssd7120-overview.htm

https://www.intel.com/content/www/us/en/products/memory-storage/solid-state-drives/gaming-enthusiast-ssds/750-series/750-800gb-2-5-inch-20nm.html

U.2 Hardware RAID card

File:Images/image10.png[edit]

U.2 Drive Cable

File:Images/image11.png

U.2 Compatible SSD

File:Images/image14.png

Storage is a python dictionary associated with any operator, used to keep user-specified data within the operator.