Data Center Post | Egress Expenses: Minimize Surprises And Maximize Data Availability

The National Aeronautics and Space Administration (NASA) is extraordinary in many ways. America’s space agency is an expert at defying gravity. Data gravity—the consequence of attracting and using additional applications and data—proves to be a bigger obstacle. NASA is facing many of the same challenges that other organizations here on Earth are addressing.

In the coming six years, NASA’s Earth observation data is estimated to skyrocket from 32 petabytes to 247 petabytes, as noted in an internal audit released in March 2020. This exponential increase is tied to “upcoming high-data-volume missions.” Data from these missions is being migrated from NASA’s 12 Distributed Active Archive Centers (DAACs) to a project known as Earthdata Cloud, which is operated by Amazon Web Services (AWS). When users—the scientists analyzing NASA’s data—download data, egress charges (costs assessed for moving data from a cloud) are billed to the agency.

But can NASA cover the egress costs? Transferring 247 PB of data a single time at an estimated cost of $0.05/GB would cost $12.35 million. Transferring data once isn’t the same as repeatedly writing it out over time, though, which may contribute to the audit’s estimate that the related cloud budget is projected to grow exponentially in the coming years, reaching approximately $30 million annually by 2025. The audit also found that “current cost projections may be lower than what will actually be necessary to cover future expenses and cloud adoption may become more expensive and difficult to manage. Collectively, this presents potential risks that scientific data may become less available to [researchers] if NASA imposes limitations on the amount of data egress for cost control reasons.” NASA is still in the process of identifying which data sets will migrate to Earthdata Cloud, indicating that it will continue to struggle with its cloud usage projections.

Egress Charges Can Impede Innovation

A multi-cloud approach combines different cloud services supplied from more than one cloud provider. These cloud providers could be public clouds (like AWS, Azure, or Google) or private clouds. Public cloud services are increasing steadily, with hundreds of services that can differentiate one cloud from another. A multi-cloud strategy facilitates innovation by enabling developers and researchers to select the services (e.g. for security or streaming analytics, predictive modeling, unstructured data mining, and much more) from each cloud that are the most suitable tools for their apps. This can foster collaboration in a variety of industries, with applications ranging from genomic sequencing to geospatial land surveying and the creation of video content in the entertainment industry.

The egress charges associated with actually accessing data can significantly impede efficient multi-cloud usage and collaboration. Ultimately, the charges restrict data availability and multi-cloud functionality, creating barriers to sharing data across platforms.

Take These Steps for Savings

NASA’s challenges with accurately estimating the total costs of storing and accessing data in the cloud aren’t unique. The following steps can help you make the most of a multi-cloud approach without overspending:

• Optimize costs based on performance needs: Match less frequently-accessed data with more cost-effective storage and use high-performance storage for data-intensive workloads with flexible, tiered design.
• Build in flexibility from the start: Cloud-attached storage with low-latency, high-throughput connections can allow your teams to utilize services like AI and data analysis from multiple clouds simultaneously without moving data around. It also scales when you need it, so you’re not paying for more storage than you need today.
• Calculate accurate cost projections: As part of the cost estimation process for your migration to multi-cloud, calculate both your anticipated data storage needs and the costs for making it available to multiple teams of users. Consider centralized storage that makes it easy to share data with different apps and clouds. Choosing the right storage tier, the right cloud platform, and anticipating egress fees are all part of a thorough budgeting exercise.
• Shop for Deals: Costs associated with moving data out of a public cloud can add up quickly. Egress expenses vary across the public clouds and are based on the amount of data (measured in gigabytes) being moved out. To reduce egress fees, consider these options:
  • Each cloud has its own egress fee structure. Use the least expensive option when the services you are accessing are comparable; use the more expensive cloud only for the workloads that require it.
  • Direct replication between on-premises and cloud-attached storage that bypasses the public cloud can avoid incurring egress fees.
  • Earlier this year, Dell EMC Cloud Storage Services eliminated egress charges for data written to Isilon from Azure.

Thorough upfront planning for cloud migrations can help minimize surprise costs, application refactoring, and rework. This effort allows your organization to capture cloud cost savings while making your data accessible to the teams that need it.

Rebekah Dumouchelle is the Sr. Product Marketing Manager for Faction. With more than 15 years in the tech industry, Rebekah’s focus is on answering how technology can help enterprises solve their business challenges. Rebekah holds a B.S. in Computer Science and an MBA.

For the original version of this article, please visit datacenterpost.com