Why Process-to-Process Visibility Is So Important
April 12, 2019
One of the challenges in securing processes operating in cloud environments is the visibility limitations imposed either by the cloud service provider or the customer’s solution architecture.
In the case of the cloud service provider, because the environment is shared by everyone who has access to that cloud service, including many thousands of their customers and millions of users, there is only so much network activity the provider will allow you to see and monitor. For example, to help monitor network traffic within your cloud network, several service providers offer flow logs which enable you to log traffic through all network interfaces in your environment. This is useful for identifying potentially malicious traffic, traffic that fits the signature of known threats, and traffic that violates defined policies. However, flow logs only show you so much. Because they log traffic between network interfaces, they may miss inter-process data transfers. Also, they do not offer visibility at the packet level, and they do not allow content inspection. These restrictions limit the value of these kinds of logs for intrusion detection.
Then you have to consider your solution architecture itself, which in cloud native applications, relies heavily on containerized services. With containers, you can easily deploy functionality, whether it’s complex applications or microservices, in containerized environments that are inherently secure. However, this architecture can limit visibility. For instance, a network intrusion detection system might be able to monitor traffic to and from an EC2 instance, but it would miss activity between multiple containers sharing data inside one EC2 instance. To further complicate activity monitoring in this environment, container instances come and go on-demand, as can EC2 instances. Some microservices, data caches, or temporarily assigned IP addresses may only be active for a few minutes before being deleted. Any activity not logged during that brief service period is unavailable for analysis and forensics.
These limitations in visibility can leave you open to certain kinds of attacks. For example, cron is a utility that runs jobs based on a pre-established schedule. A cron command configured to launch a job that utilizes an S3 bucket would run without any error if it found an appropriately named S3 bucket being operated by a data thief. Without adequate visibility into what is actually happening with that data, there would be no indication of a problem.
Another case would be the compromise of a container service deployed inside an EC2 instance. For example, Docker Compose allows you to run multiple containers inside a single E2C instance in such a way that they connect to perform different functions needed to deliver the service. You might have one container with a Python container working with a Redis container to perform a function. Redis, a popular, freely available in-memory data structure, is widely used in applications for page and session cashing, storing messages, and in many other ways. Although Redis is used extensively, security researchers have recently found that 75% of deployed Redis servers show signs of malware infection. If your security strategy depends on network intrusion detection tools that have no visibility into inter-process activity inside the E2C instance, a lot of malicious activity can go undetected.
The solution to these potential gaps in cloud security is one that monitors and logs all inter-process activities, even those occurring inside the same file. You need a host-based intrusion detection system designed to monitor process hierarchy, process and machine communications, any changes in user privileges, internal and external data transfers, and all other cloud activity. An effective system looks across all layers, and it analyzes activity based on normalized behavior, which gives a continuous real-time view even across short-lived services that may only exist for a few minutes. Having that process-to-process visibility is a critical factor in having strong, effective security built into any cloud environment.