FinOps for Cloud Cost
Management & Optimization

As discussed above, FinOps.org is championing cost awareness, cost management and optimization capabilities, and the cultural transformation necessary to control and manage cloud costs. One such capability in an early maturity stage is workload management and automation. Here, the primary strategy is to tag workloads appropriately to trigger automated workflows and create dashboards and alerts when costs breach budget thresholds. For example, a workflow can be triggered to shut down non-production instances outside of business hours. We believe that more advanced workflow capabilities are required that consider the type of application running in the cloud. In addition, for the IaaS class of applications, it is necessary to consider application instance-level performance characteristics in making decisions that impact cost.

Cloud vendors offer tools to help companies manage cloud costs. This generally takes the form of cloud cost optimization recommendations, wherein a recommendation often links to underutilized cloud resources. While this approach may work in many cases, implementing some recommendations is often impossible due to application availability and performance considerations. What is missing is that cloud vendor tools lack application context. Cloud vendor tools don’t know what application is running on the infrastructure and, more specifically, the application performance and availability characteristics. This gap drives the need to implement cost optimization recommendations that are application-aware.

Application-aware optimization recommenders can work by storing application performance and availability characteristics for each application instance and then using this information in one of four ways: (a) looking at cloud vendor optimization recommendations and deciding on what actions to take based on application instance performance characteristics; (b) apply AI/ML anomaly detection to cloud resource performance data along with application instance performance characteristics to decide what needs to be done; (c) obtain live application performance characteristics data from an APM (application performance management) tool or cloud vendor tool (e.g., CloudWatch Application Signals) and compare that to the defined application instance performance characteristics; or (d) Use all three methods defined above, namely (a), (b) and (c). In this paper we focus on the first approach due to its simplicity – it is easy to import cloud vendor recommendations.  Cloud vendor recommendations are also based on AI/ML techniques such as anomaly detection.

In fact, at the most recently concluded 2023 AWS re:Invent conference, there was an announcement about a new feature known as CloudWatch Application Signals. (9) Application Signals will allow users to monitor application metrics like latency, faults, availability, etc. The feature is currently not generally available, but it aligns with our notion of application awareness in cloud workloads.

Companies run many different types of “applications” in the cloud. Sometimes, it may even be difficult to pinpoint the application running in the cloud. However, if a company is getting some business value from using a cloud service, it should be possible to associate the cloud cost with an application instance. The natural question is, what are the different types of “applications,” and can you apply application-aware cost optimization to all of them? To answer this question, we looked at two things:

• Common application performance and availability characteristics
• Taxonomy of Software Types (7)

Application architectures have evolved over the years from mainframe to client server to service based and finally microservices based architectures. In looking at application performance characteristics we’ve excluded the mainframe class of applications. Some of the most common types of application performance and availability characteristics include:

• Average response time: this is the time taken on average to respond to all client requests over a period. A given application may have a value of a few seconds, less than a minute, or even minutes, indicating what is acceptable in the context of the application instance.
• Latency: measured in milliseconds, it is the delay between a user’s action on an application and the application’s response to that action.
• User satisfaction Apdex score (https://www.apdex.org): This is calculated based on the response time that is considered acceptable for the application. There are three types of users: Satisfied, Tolerating, and Frustrated users. Apdex = (SC + TC/2 + FC × 0)/TS, where SC is the count of satisfied users, TC is the count of tolerating users, and FC is the count of frustrated users. TS is the total number of samples collected.
• High Availability Required: Yes/No.
• Max CPU usage: this is the desired maximum CPU usage across all servers used by the application. CPU usage above the maximum implies there is a potential need to scale up.
• Max Memory usage: this is the desired maximum memory usage across all application servers. Memory usage above the maximum implies there is a potential need to scale up.
• Min CPU usage: this is the desired minimum CPU usage across all servers used by the application. CPU usage below the minimum implies there is a potential need to scale down.
• Min Memory usage: this is the desired minimum memory usage across all application servers. Memory usage below the minimum implies there is a potential need to scale down.
• Minimum free disk block storage.
• Application Instance Environment: PROD, UAT, QA, TEST, or DEV
• Does the application use cluster orchestration software? For example, is the Application running in a Kubernetes cluster: Yes/No.

Notice that 5-9 are resource-level characteristics more than application characteristics. However, cloud-based cost optimization recommendations are generally specified at the resource level. Hence, we need to break down the application characteristics to the resource level. For example, if the application has three different virtual machines running the web server, database, and business logic, then the Min. CPU usage applies to any of these virtual machines, including any redundant virtual machines in the application cluster. The other characteristics are true application level and can help resolve scenarios in which resource-level data is unavailable, insufficient, or inconclusive. 

For many applications running in the cloud, it should be possible to specify some or all of the above performance and availability characteristics. The cloud deployment model of the application – Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), or Network as a Service (NaaS) is also a factor. SaaS applications are consumed as subscriptions with a fixed cost per subscription per month (or quarter or year). So, depending upon the number of active subscriptions, you get a fee that only varies by a small amount per month. However, IaaS, PaaS, and NaaS deployments can result in highly variable costs per month depending upon usage. In addition, PaaS can have a baseline minimum fee plus specialty licensing costs over and above the usage-based cost.

Companies large and small, across all industry segments, and government agencies have adopted cloud services. As a result, to develop a comprehensive and complete cost optimization capability it is necessary to build workflows for the full range of application types that run in the cloud.  By analyzing the different application types, we can also determine if the defined application performance characteristics can be applied to all of them, or if additional characteristics need to be defined.

There are four major categories of software and multiple sub-categories under each: (7)

Data-dominant Software

Consumer-oriented software: most of these are software applications that run on client devices, although many are also gradually migrating to the cloud. For example, word processing, spreadsheets, training/courseware, etc. When such applications are hosted in the cloud, they fall under the category of Business-oriented software (see below) offered as a SaaS subscription to company employees. From a company perspective, there is a fixed monthly cost based on the number of subscription licenses. From the perspective of the company offering/hosting these SaaS services, it could assign application performance characteristics to it.

Business-oriented software: this class of software is the most impacted by the cloud, and over time, more and more of these are moving to the cloud. This class of software includes transaction processing systems, data warehouses, corporate management systems, etc. For all these types of software, it should be possible to assign application performance characteristics. However, when any of these are offered as a SaaS subscription by a vendor, it becomes a fixed subscription license cost per month for the consumer. At the same time, for the provider/host, it would be possible to assign application performance characteristics.

Design and engineering software: many of the software tools in this category run on the client computer and tools, but once again, like consumer-oriented software, many of these are also moving to a SaaS model, and the same considerations as outlined before will apply in this case as well.

Information display and transaction entry: This includes search engines, news and media, weather, maps, e-commerce, e-finance, e-government, etc. Generally, from a client perspective, such services are free or charge a SaaS subscription license. From the provider’s perspective, however, it should be possible to assign application performance characteristics.

Systems Software

Operating systems: this class includes OS kernels (like Linux distributions) and virtual machines, among other related tools. Many of these tools are either free or offer a subscription license. However, virtual machines are a classic example of an IaaS cloud service. A company that is using virtual machines must decide on the application or team that is getting some business value from it, and then, based on that, determine what performance characteristics to assign to each virtual machine.

Networking/Communications: this category includes software that is generally part of the operating system bundle and is not charged separately.

Device/Peripheral drivers: this category includes software that is generally part of the operating system bundle and is not charged separately.

Support Utilities: mostly utilities like backup/recovery, network traffic monitoring, etc. Generally, these are tools purchased as SaaS subscriptions or one-time fee.

Middleware and System Components: includes database servers, virtual machines running middleware, etc. When consumed as IaaS, assigning application performance characteristics to each of these should be possible.

Software Backplanes: this includes platform as a service (PaaS), which generally requires a subscription license fee (based on the number of users) plus usage-based cost. It should be possible to assign application performance characteristics to each of these.

Servers: this includes load balancers, email servers, proxy servers, etc. When consumed as an IaaS service, it should be possible to assign application performance characteristics to each of these.

Control-dominant Software

Software of this type is several layers removed from the application layer and not used as is. The exception is process control software, including industrial, air traffic, or nuclear plants. However, such software is generally not run in the public cloud due to security concerns.

• Hardware control
• Embedded software
• Real-time control software
• Process control software

Computation-dominant Software

This category of software is often an ideal candidate for running in the cloud due to the need for scalability. For all such applications, it should be possible to define application performance characteristics.

• Operations research
• Information management and manipulation
• Artistic creativity
• Scientific software
• Artificial Intelligence

In summary, other than control-dominant software, most other software can be run in the cloud. However, many of the applications that run in the cloud can be purchased as SaaS subscriptions, which translates to a fixed amount per month based on the number of active users. SaaS software costs, although generally fixed per month, also need to be managed on an on-going basis as employees leave the company. Indeed, the amount of money wasted on un-used licenses is quite significant as, on average, only 56% of SaaS licenses are used. (8) The identified application performance and availability characteristics can be defined for each application instance that runs on IaaS/PaaS/NaaS resources, from simple virtual machines to advanced transactional applications. We suggest that for each of the identified software categories of applications that can run in the cloud a set of “default” performance and availability characteristics be defined. The default characteristics can then be used to specify (or override) application-instance specific values. Better optimization decisions can be made based on these definitions and input on actual performance data or cloud vendor optimization recommendations.

As mentioned earlier, we can define a cost optimization workflow that leverages cloud vendor optimization recommendations. See Figure 4 below for an example workflow that improves upon cloud vendor recommendations. In this workflow, only resource-level application characteristics have been used. However, it is important to note that resources are first mapped back to the correct application instance to load the corresponding set of application performance and availability characteristics.

Figure 4 Application-Aware Cost Optimization Workflow

Workflows such as the workflow in Figure 4 can be quite easily implemented in a cost management and optimization platform. Such a platform not only stores the application performance characteristics of the different application instances under management but also reacts to cloud recommendations by executing the workflow. The platform must support multiple different optimization workflows depending on application software category and the approach used for application-aware cost optimization. These and other details of such a platform are described in the next section.

The best approach that offers a means of implementing and institutionalizing FinOps capabilities in the various domains is to deploy a cloud management platform. Institutionalizing such capabilities makes it possible to scale such practices across the entire enterprise. Platforms serve as a foundation on which companies can build over time. The costs and challenges of implementing a FinOps practice scale up with the size and complexity of the cloud ecosystem. So, while cloud native tooling or basic infrastructure as code may be suitable for small teams, enterprise-scale multi/hybrid cloud environments require the support of additional IT tooling and human resources. Platforms are better suited for larger organizations because of their ability to handle interdependency issues, facilitate automation, and institutionalize best practices. It can also be cheaper for transitioning organizations to implement a pre-built system instead of creating one from scratch.

The type of cloud ecosystem management platform that would be useful in accelerating the adoption of FinOps principles would need to have a few key features: a governance structure, a self-service catalog, workflow management, a database, and integration capability. A platform must have a governance model that can align with the business organization. It should include a multi-tier structure that enables operators to define which groups in the enterprise have ownership of which cost centers. It should have some means of assigning and enforcing budgets. The governance structure also serves as a mechanism for siloing data related to the cloud services so that users can only interact with services relevant to their roles. This governance model becomes extremely important when allocating cloud costs. The service catalog amplifies the benefits of the governance structure. Services acquired via the catalog can have tags assigned automatically to match the resources to a cost center, facilitating show-back, a function many people still rely on spreadsheets to perform. (1) Additionally, budgets can be enforced by restricting ordering as they approach their budget limits, reducing the risk of rapidly ballooning cloud expenses.

The self-service catalog acts as a cloud-agnostic system of engagement for users acquiring cloud resources, allowing the catalog operators to productize the services. The offers on the catalog should be designed with enough flexibility to accommodate the needs of developers. Still, the default settings for all services should be packaged to take advantage of savings plans and apply cost-saving automation to ensure resources are only active when used. This sets up an opt-out scenario for cloud consumers where best practices can be built into the service itself. Also, when users are restricted to acquiring services through a catalog, it can act as a kick-off point for procurement flows, including approvals and budget enforcement.

A cloud ecosystem management platform can create a universal service layer for much of the tooling to deliver and manage cloud services. Its database and ETL processes can normalize data from public and private cloud sources and coordinate the delivery of services within the business context of the users procuring the cloud services. Businesses are deploying an increasing number of tools to manage the finances of their cloud operations. Still, the FinOps Foundation survey results referenced above show that only 16.4% of the respondents have automated workflows with finance integration! The bottom line is this: modern cloud ecosystems require a large variety of tools to manage. (1)

A business with many employee groups, particularly when combined with varied product and service lines, faces competing priorities and needs. Major technology companies employ thousands of people, with the biggest companies employing hundreds of thousands. Only a percentage of those will be cloud practitioners, but these numbers should give you a sense of the scale of the challenge. Platforms serve as the foundation for a hub and spoke system of integrated services by using workflow automation, data normalization, and REST APIs to facilitate communication between cloud resource and finance management tools.

Let’s look at how interacting with a platform can help different sets of users engage with a FinOps practice.

Application Developers want to create optimized cloud-native solutions using the most efficient resources available that can achieve their goals. They are focused on writing/testing software code and want access to agile cloud computing resources that can be manipulated as they test and discarded when they are no longer needed. They are invested in minimizing their activity’s impact on budgets but can also generate a lot of variable costs depending on their needs. They also desire to be self-service because time spent procuring cloud resources prolongs the delay between ideation and execution. They are interacting with DevOps tooling for managing code changes, automating testing, and automating code pipelines on a regular basis. A persistent challenge for FinOps practices is empowering this type of engineer to utilize the cloud services they need while also optimizing costs.

An application developer does not have time, nor does their job incentivize them to allocate attention to cost optimization. They are better positioned to optimize resource utilization according to the functions required of a given cloud asset. They are most successful when empowered to iterate and fast fail in a flexible environment, so cumbersome procurement processes with multiple manual steps are detrimental. When they interact with a cloud management platform, it is to monitor/modify existing resources and request new resources. Platforms help them by offering dashboards that aggregate performance and utilization data across clouds and by creating a simplified catalog experience for highly automated procurement. Well-defined catalogs speed up the development process by enabling developers to modify service variables in a click interface with built-in checks for technical and financial requirements. In addition to time-saving measures, platforms handle the allocation of service costs automatically based on the identity of the user paired with governance details. They can also encourage cost ownership in developers, the number 1 FinOps challenge in 2023 (1), by connecting cost models directly to the cloud services in the catalog and by including cost trend charts alongside the utilization metrics in dashboard views.

This helps to inform engineers of possible costs up-front and lets them compare the cost of services with the value they bring. Engineers typically have the contextual technical information required to make a quality evaluation, so it is highly beneficial to organizations to provide them with financial context. Order workflows automatically update the various financial and IT systems and further reduce the time it takes to add or modify cloud services.

Infrastructure Developers want to design an optimized cloud architecture for use by application developers and service delivery teams. They are focused on writing infrastructure as code that is efficient in terms of cost, and efficient in terms of useability. They want to create reusable blueprint assets that can be used to institutionalize security practices and design principles and have some limited variables that can be adjusted to fit specific use cases. Infrastructure designed by these engineers will also be used as a component in different stages of automated CI/CD pipelines. They are invested in learning and deploying new cloud service architecture as it becomes available on hyper-scaler catalogs and in identifying and utilizing savings opportunities present within a service provider’s system.

A cloud management platform can help infrastructure developers by helping get their designs into the hands of consumers quickly. The blueprints for new services can be stored and/or published on the self-service catalog, regardless of which cloud they have been designed for. Moreover, time is saved by re-using platform services like workflow automation, so the business can onboard new service types almost as quickly as they become available. New integrations are established once for a particular service type, with subsequent deployments of a service variant using the original integration. Essentially, any type of API-enabled cloud service can be integrated with the platform. While the vendor will regularly add support for new service types, platform operators should be able to integrate their own service types using a software developer’s kit. The catalog also offers an opportunity to create highly defined or productized services. The service definitions can include abstracted technical variables that reduce error, and they can include business variables that help update external tools and control/allocate costs.

Tooling Developers create the CI/CD pipelines executed by the application developers. The pipelines include multiple stages running from sourcing code through the deployment of certified and tested code into production environments. Ideally pipelines are fully automated, though many are likely to include manual transitions between stages depending on how mature a product or solution is. These developers want to ensure that the adaptation of pipelines for a variety of services is simple and free from error.

Cloud management platforms help tooling developers by connecting elements of the pipeline together and reducing the complexity of updating pipeline variables for different service lines and environments. Pipelines can be configured as offers available on the same catalog used to deploy infrastructure, creating an opportunity to deliver DevOps as a service. Pipeline offer variables that could include updates to scripts or even the infrastructure the new code will deploy on. The business benefits from the same productization process as infrastructure services and can connect pipelines to broader workflows, price models, and showback automation.

Service delivery teams are responsible for deploying and maintaining solutions and infrastructure to their end-users. End-users could be internal or external to the business, and these employees are interested in delivering services that are compliant with user needs while adhering to business processes that guarantee service uptime and performance. They use tools to monitor support requests, cloud resource utilization, and tools to interact with cloud resources for change management and issue remediation. These employees are able to identify over/under-provisioned resources that developers and architects should review for optimization. They also look for methods of reducing the level of effort and response time required to fulfill service requests.

Cloud platforms can transform the service delivery model of the cloud completely by managing orchestration and providing an interface for requesting services. The benefits of cloud catalogs have already been discussed, but price modeling can have major implications for service delivery. The true cost of delivering cloud services includes more than just the compute/hour rates charged by public cloud vendors. There are also licenses and labor considerations, and platforms allow their operators to include those costs on offers in their catalog. This is especially beneficial for companies with large private cloud footprints seeking to capitalize on heavy investments in hardware and data center infrastructure. Additionally, platforms enable service delivery teams to monitor resources spread across multiple and hybrid cloud ecosystems. This, combined with true cost price models, empowers teams to make business value-driven decisions when right-sizing resources or deploying resource-based savings plans.

Project managers and application owners coordinate the day-to-day activities of the teams responsible for developing and maintaining software solutions and products. They are concerned with creating and maintaining business processes that enhance communication, reduce time creep, and restrict budget bloat. They are also responsible for cataloging and reporting time, requirements, and costs for their project. They closely monitor how costs over time compare to budgets and look for ways to reduce spending without compromising service quality.

Platforms empower these users by channeling the procurement of cloud services through a controlled engagement system. The governance model gives them direct management over their team’s resources and cost roll-ups across cloud accounts and environments. If their teams place orders through a platform catalog, they are assured that costs are properly allocated to their cost centers. Managers can enforce budgets and determine how much autonomy to give their teams by assigning workflows to match different service scenarios, including approvals at various budget thresholds. Finance and budget dashboards on the platform, along with notifications, keep stakeholders informed of changes in expenses before they become major problems. The rules established for previous projects can be copied and modified as new projects arise, helping to retain lessons learned and improve outcomes over time.

Finance Managers and Executives are primarily responsible for evaluating the business value of cloud services and identifying areas to increase or reduce budgets. They are interested in monitoring cost trends, predicted costs, and the overall health of the cloud practices under their responsibility. They also want to create/nurture sustainable long-term strategies and practices that benefit the business. This means identifying future risks and opportunities, planning for them, and informing and enforcing the overall strategy across numerous teams within the business. They use tools that translate cost/utilization data into actionable information and tools that mandate compliance with company and legal standards. They also want tools to make their business lines more adaptable and responsive to change and opportunity.

Cloud management platforms are typically provider-agnostic by nature. This helps companies maintain technological independence from specific vendors, and accelerated onboarding of new services keeps companies agile. The governance structure and robust allocation tools in a cloud management platform help aggregate data across the entire cloud ecosystem, increasing visibility into where spending should be monitored closely and exposing opportunities for increased investment. Cost optimization recommendations from cloud providers can be helpful, but they are given without full context. Platforms aggregate the recommendations and can optimize them through application-aware workflow management. This enables human or artificial intelligence to make more informed decisions when assessing overall cloud utilization. More broadly, these users benefit from using technology to encourage broad use and adoption of FinOps practices across the entire organization. Cloud management platforms help orchestrate interconnected systems while reducing the need for additional training because they enable organizations to convert best practice ideas into executable programs.

There are more user stories beyond the scope of this discussion. Platforms can have a positive impact on security and regulatory compliance. They can greatly reduce the time it takes to get new products to market or start new practices within a business. The greatest strength of cloud ecosystem management platforms is their efficacy when dealing with interdependent systems. Given that IT tooling complexity is increasing, these challenges of system orchestration will only grow over time. Having a platform to build on as a foundation can help prepare a company to take advantage of new cloud services instead of being overwhelmed by them.