What are the measures of size and complexity of IT infrastructure

Question

How can I compare different infrastructure in regards to size and complexity. What can I measure and compare e.g. number of nodes, number of servers, architectures etc.

How do those measures and variables differ in what they compare? Which of them make sense to compare and what is the cardinality at which there is actual difference in the kind of work done, rather than just a bit of extra work.

What will let me decide that "This infrastructure is very large" or "This infrastructure is very complex" and what is the difference?

Answer 1

TL;DR: Measures of size can be split roughly in three different categories that I would define as depth in what you manage vs. what you outsource or consume as a service, breadth of services supported and height in numbers of instances,servers and customers. Measures of complexity largely depend on selected systems architecture, organizational structure of people supporting it and skill sets required. Where depth and skill sets go hand in hand adding both to size and complexity.

Size mesures

Note: Most of the following size requirements can also add to complexity through systems architecture, skill sets and organization structure needs.

Depth of Infrastructure

How much of the infrastructure you are outsourcing to others in increased levels of depth:

1. Do you simple use Software as a Service for all that you do?
2. Do you operate fully in public cloud, private or hybrid cloud or using some PaaS?
3. Do you use Infrastructure as a Service?
4. Do you use Hosted and Managed Infrastructure in rented DC space?
   • Do you own or rent the hardware?
   • Does the provider manage infrastructure monitoring?
   • Does the provider manage basic system administration?
   • Does the provider manage hardware failures and maintenance?
   • Does the provider manage racking and server installation?
   • Does the provider manage internal networking?
   • Does the provider manage internet connectivity and routing?
5. Do you have just data centers with contracted remote hands?
6. Do you host everything on premises or in your own datacenters?

Breadth of Infrastructure

What are the different types of services you support?

• Compute Resources
  • Bare metal servers
  • Virtualization layer (VMWare)
  • Container layer (docker, k8, mesos)
  • Serverless layer (lambda, functions)
• Storage Resources
  • Standalone storage appliances
  • RAID in servers
  • Large standalone relational database clusters
  • Time series databases
  • Object store clusters
• Network Resources
• Observability Resources
  • System Log servers
  • Metrics and Graph systems
  • Search clusters
• Automation Resources
• Backup/Recovery Resources
• Complex compounded services
  • ELK, Hadoop, etc.

Height of Infrastructure

• What is the scale for each resources that you need? Do you operate the services on single server/instance or do you need use clusters of machines?
• What is the level of redundancy you need?
• What are your availability requirements?
• What are your requirements for latency and throughput of services?
• Do you need geographically distributed infrastructure? (international business, latency requirements or regulation compliance like GDPR, data localization laws, etc)
• Do you need multiple datacenters in each geography?

Complexity Measures

Just very briefly...

System Architecture

When it comes to Infrastructure complexity, it quite closely follows the complexity of the distributed systems supported by the infrastructure. You have to take into account two types of systems:

1. Distributed systems supporting individual services.
2. Distributed system created by interdependencies of the services.

Distributed Systems Complexity

Every service your infrastructure supports can have in and of itself a different level of complexity with varying levels of requirements on the infrastructure. Systems supporting services can range through:

• Single threaded.
• Multi-threaded (shared memory, shared disk)
• Parallel systems with data sharding
• HA Failover (Primary / Standby) (Cold, Warm, Hot)
• HA Cluster (N+M)
• Real-time Clusters

Interdependence of Services

Let me start with an example. Let's say your infrastructure reports test results into ElasticSearch cluster. Your pager depends on monitoring and test data provided by ElasticSearch. Geographical distribution of ElasticSearch cluster makes it depend on your datacenter networking connectivity. Now one of your internet providers decided to make an unannounced maintenance on Saturday night, throughput drops, your traffic is rerouted to backup provider, monitoring traffic is de-prioritized to customer data traffic, ingest of monitoring events slows and your pager goes crazy.

Every time two services, two parts of the infrastructure depend on each other, they create a new single distributed system, whose complexity should be judged independently. Such dependency can be either removed or reduced. Remember that system is only as redundant and as available as the intersection of all services that it depends on.

Further examples of complexity increasing factors:

• Dependencies on external services.
• Attempts to mitigate failure from service dependency.
  • Multiple providers
  • Caching of data

Organizational structure

This is a chapter in and of itself ... people are often overlooked part of the overall system of IT infrastructure. We rarely think about redundancy, availability and latency factors when it comes to people, but just as with computers, these same issues affect organizations maintaining infrastructure and the complexity of it can sometimes easily outweigh the complexity of the computer systems. People involved in maintaining your infrastructure can span multiple time zones, languages, geographic locations, companies, pay scales and legal codes. Any of those factors are signs of increase in complexity.