Logging and Viewing System Logs: journalctl Commands

1. Why work with logs?

Logs are like your personal "detective", recording every important operation in the system. They capture actions of the kernel, applications, services, and other system components. Logs are an essential tool for troubleshooting and monitoring system performance.

Imagine your server suddenly stops responding. Instead of guessing, like Sherlock Holmes, what went wrong, you can just peek into the logs — it's all laid out there. For instance: "The ssh service crashed", "Disk is full", "The cat walked across the keyboard and stopped an important service". Logs will tell you everything.

Working with system logs helps you:

1. Diagnose issues. If something's going wrong, you can find the error or warning in the logs.
2. Track service operations. For example, find out why the PostgreSQL database stopped or when the web server was last updated.
3. Collect analytics. Monitor the performance and metrics of services.
4. Enhance security. Logs can show you SSH break-in attempts or unauthorized access.

---

2. Introduction to journalctl

journalctl is a powerful command for viewing and filtering system logs in Linux. It works with systemd-journald — a system for collecting and storing logs managed by systemd.

Basic operations with journalctl

First of all, let’s see how to use this tool. Open up your terminal and type:

journalctl

This command will show you all the system logs sorted in chronological order. If you expect to see something minimal, brace yourself: logs usually occupy several screens. Don’t freak out, we’ll learn how to filter them a bit later.

Now, to exit log viewing, press q (did you notice the similarity between less and journalctl?).

---

Filtering Logs by Time

Now let’s magically narrow down the logs to a more manageable range.

Viewing logs from the last hour

journalctl --since "1 hour ago"

Viewing logs for a specific day

journalctl --since "2023-10-31"

Specifying a time range

Want to see events from 8 AM to 9 AM? No problem:

journalctl --since "2023-10-31 08:00:00" --until "2023-10-31 09:00:00"

Time filtering is useful, for example, if you know the problem started at a specific moment. Analogy: you don’t read the entire book to figure out who the murderer is — you just check the last pages.

---

Working with Logs of a Specific Service

Instead of looking through the entire system, you can focus on a single service. For example, want to see how the nginx web server is doing:

journalctl -u nginx

If you want to see entries only from the last hour (because we all love fresh info):

journalctl -u nginx --since "1 hour ago"

Quick list of examples:

• -u ssh: Logs for the SSH service.
• -u cron: Logs for CRON tasks.
• -u mysql: Logs for the MySQL database.

---

Reading Logs in Real-Time

When you’re fixing something live, it can be helpful to watch the logs in real-time — like watching a football game. Use this command:

journalctl -f

The -f option works similarly to tail -f, showing new log entries as they appear.

For example, try running this command while restarting some service in parallel:

sudo systemctl restart ssh

You’ll see changes — a log entry about the restart will pop up.

---

Searching for Errors in Logs

Logs are often long, and looking for the info you need manually feels like finding a needle in a haystack. So you can use keyword search.

Basic usage:

journalctl | grep "ERROR"

This example will show only those lines that contain the word ERROR. You can replace ERROR with anything, like WARNING, failed, or panic.

---

3. Special Options for journalctl

Viewing Entries Only After Reboot

Wanna know what happened after the last system boot? Sounds intriguing? Just do:

journalctl -b

Every system reboot is logged as a separate "block". If you wanna check out the logs after the second-to-last boot (yep, sometimes that's handy):

journalctl -b -1

Filtering by User or PID

Sometimes, you need to track the actions of a specific user or process. For example:

journalctl _UID=1000

Replace 1000 with the user's ID (you can find the ID using the id command).

To filter logs by PID (process ID):

journalctl _PID=1234

---

4. Practical Examples

1. Searching for SSH service errors.

   You wanna figure out why SSH refuses to connect. Use:

   journalctl -u ssh | grep "ERROR"
   

2. Analyzing logs for the past 24 hours for CRON.

   Say you're checking why your daily script didn’t execute:

   journalctl -u cron --since "yesterday"
   

3. Real-time monitoring of the web server.

   Web server under load? Check the logs for errors:

   journalctl -u nginx -f
   

Practice Exercise

1. Find error logs for the SSH service in the past 24 hours.
2. Using journalctl, display logs for CRON only for the morning. Make sure the command works with the correct time range.
3. Set up real-time monitoring of systemd logs and start a few system services to see their status in the logs.

Logging isn’t just about error directories or troubleshooting. It’s your secret agent that helps keep an eye on the entire operating system. Once you master the journalctl command, you’ll have a powerful tool for analyzing, optimizing, and ensuring the stable operation of a Linux system under any conditions.