What is machine data?

Machine data is information automatically generated by computer software in the form of a log message, debug message, error message, or status message, as a response to an event. Usuallly the information was generated for the purpose of troubleshooting computer software and computer hardware. Machine data includes informative information about the performance of an operating system from services and processes like package management, resource utilization, and network management.

AWS Logs

Informative data from support service monitoring, alarms and a dashboards for metrics, and can also track security-relevant activities, such as login and logout events.

Authentication

Authentication data can help identify users that are struggling to log in to applications and provide insight into potentially anomalous behaviors, such as activities from different locations within a specified time period.

Firewall

Firewall data can provide visibility into blocked traffic in case an application is having communication problems. It can also be used to help identify traffic to malicious and unknown domains.

Network Statistics

Network statistical data can provide visibility into the network's role in overall availability and performance of critical services. It's also an important source for identifying advanced persistent threats.

System Logs

System logs are key to troubleshooting system problems and can be used to alert security teams to network attacks, a security breach or compromised software.

Web Server

Web logs are critical in debugging web application and server problems, and can also be used to detect attacks, such as SQL injections.

Sensor Data

Sensor data can provide visibility into system performance and support compliance reporting of devices. It can also be used to proactively identify systems that require maintenance.

Comments

The meaning of time in reinforcement learning

Reinforcement learning (RL) is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning is concerned with how software agents ought to take actions in an environment in order to maximize the notion of cumulative reward through the process of trial and error. In reinforcement learning an agent starts at an empty state then analyzes the available datasets according to a policy of positive states and negative states. Rather than being explicitly taught as in supervised learning the correct set of actions for performing a task, reinforcement learning uses rewards as signals for positive states and punishments as signals for negative states. The agent obtains the best path to a desirable reward as a cumulation of positive states and negative states. As compared to unsupervised learning, reinforcement learning is different in terms of goals. While the goal in unsupervised learning is to find similarities and differences...

Threat hunting polymorphic malware in Linux with Python

You can investigate suspicious activity that could be polymorphic malware by collecting relevant machine data from your endpoint. You can use the machine data to create your own analysis. Before you start your investigation you will need to determine normal activity on your endpoint. Normal activity is the scope of functionality of the software on your endpoint during periods of low activity and high activity. You will need some kind of software that periodically collects specific machine data from your endpoint like my software developed in Python that's available for free download at https://github.com/davidbrennerjr/server-stats-collector Ingest one or more of the following machine data from Category #1. Ingest one or more of the following machine data from Category #2. And ingest one or more of the following machine data from Category #3. Category #1 General system-wide error messages from /var/log/syslog Auditing logs of application rule...

Application behavior monitoring in Linux with Python

You can monitor application behaviors by collecting relevant machine data from your endpoint. You can use the machine data to investigate suspicious activity and create your own analysis. Before you start your investigation you will need to determine normal activity on your endpoint. Normal activity is the scope of functionality of the software on your endpoint during periods of low activity and high activity. You will need some kind of software that periodically collects specific machine data from your endpoint like my software developed in Python that's available for free download at https://github.com/davidbrennerjr/server-stats-collector Ingest one or more of the following machine data from Category #1. Ingest one or more of the following machine data from Category #2. Category #1 General system-wide error messages from /var/log/syslog Auditing logs of application rulesets Auditing logs of security contexts Auditing logs of ...

David Brenner Jr

Search This Blog