Monitoring Web Application with ELK Stack

ByKarthik Kumar D Kon19th Mar 2021, 2022-12-01T08:00:00+05:30
Read
Pause
Resume
Stop
Monitoring Web Application with ELK Stack

For Auditing & Troubleshooting issues in an application, log data is very useful. And to make it easy to read the log data via some kind of tool will make developers & maintainers happy. However, not all the log data for any event is not needed to be stored in the log record. Means, Each log record should have only the information needed for further notice or further debugging. Strictly all sensitive data should be not logged for any application.

What data to Log

In case of Drupal 8 application, which had the database logging feature within the core. Also has a logging class to write any custom logs to the database. Logging to database & querying from database is not an efficient way if we are talking about mid-sized or large applications.

A better and more efficient way to write and read logs would be using ELK stack, where Logstash ingests data from a multitude of sources in our case apache & php logs, transforms it, and then sends Elastic search. And Kibana (UI) provides an interface to search logs and also create visualizations.

Create two streams of logs, one for apache logs and other of php logs, where logs are in these locations in docker containers.

  • Apache logs /var/logs/apache2/error.log
  • Php logs /var/log/php.log

Tweak php.ini file, and add error_log=/var/log/php.log. So that logs will get stored to php.log file inside the docker container.

For logging the data in a readable format, use the following line formatter.

Whatever data you are processing to the context variable, will be in the readable format. And it's logged in as audit logs and error logs.

Send logs via Filebeat to Logstash or Elasticsearch

Use Filebeat, to move the logs created by Elastic which will tail the log file data and further send to logstash or Elasticsearch. Install the filebeat and configure it to track the log files.

In the output configuration section, the logstash host with port number is mentioned. By this the logs should be showing up on the Kibana.. If not, make sure the filebeat agent is running.

Monitoring the Logs via Kibana

On Kibana, Available fields are the indexed items which can be used as columns and also in search. You can filter logs by fields USER-ID. And each log record can be seens as a json value by expanding the _source.

You can also create Dashboards & Visualizations which can have specific data logs & also filter by fields.

26,991OFF
Hisense 139 cm (55 inches) Bezelless Series 4K Ultra HD Smart LED Google TV 55A6H (Black)

Hisense 139 cm (55 inches) Bezelless Series 4K Ultra HD Smart LED Google TV 55A6H (Black)

32,999 59,990
Resolution : 4K Ultra HD (3840x2160) | Refresh Rate : 60 Hertz | 178 Degree wide viewing angle. Connectivity: 3 HDMI ports to connect set top box, Blu Ray players, gaming console | 2 USB ports to connect hard drives and other USB devices | Dual-band Wi-Fi | Bluetooth 5.1. Sound : 24 Watts Output | Dolby Atmos | Dolby Digital. Smart TV features: Google TV | Watchlist | Google Assistant | Far Field Vioce Control | Chromecast, Miracast, DLNA | Auto Low Latency Mode for VRR | Supported Apps : Netflix, Youtube, Prime Video, Hotstar, SonyLiv, Hungama, JioCinema, Zee5, Eros Now. Display :10 bit Panel | Bezel-less Floating Display Design | ALLM | Decoding of Dolby Vision, HDR10, HLG | 1 Billion Colours | MEMC.
Televisions
350OFF
Amazon Brand - Solimo Slim Stainless Steel Water Bottle, Set of 3, 1 L Each

Amazon Brand - Solimo Slim Stainless Steel Water Bottle, Set of 3, 1 L Each

649 999
Stainless Steel water bottle set for home and office use. Made using high quality stainless steel for added durability and long life. Made from 100% food grade materials to be safe for everyday usage. Features a spill-proof design; has a snug fitting lid for easy usage and prevention of leaks. Ergonomic design for comfortable grip and rust free body. Ideal for storing water, iced tea, juices, shakes and more. Dimensions: L x W x H: 7.3 x 7.3 x 26 cm. Package Contents:- Solimo Stainless Steel Water Bottle Set of 3, Weight: 200 gm each, Colour: Silver, Capacity: 1 litre each.
Kitchen & Dining
1,272OFF
Bajaj Rex 500W Mixer Grinder, Purple

Bajaj Rex 500W Mixer Grinder, Purple

2,278 3,550
Wattage : 500 W, Voltage: 230V~50 Hz: Revolution: 20,000 RPM. Powerful 500W Titan Motor with Radio Knob: Easy selection of Speed. 2-in-1 Blade: The dry grinding jar can be used for dry and chutney grinding both. Product warranty by Bajaj - 2 Years. T&C Applied. Pulse Mode: Used for mixing and grinding without stopping your mixie. Perfect to mince chicken , get bread crumps, small puree and making lump free gravies.
Kitchen Appliances

Related Articles

32,991OFF
Hisense 139 cm (55 inches) Tornado 2.0 Series 4K Ultra HD Smart LED Google TV 55A7H (Silver)

Hisense 139 cm (55 inches) Tornado 2.0 Series 4K Ultra HD Smart LED Google TV 55A7H (Silver)

36,999 69,990
Resolution : 4K Ultra HD (3840x2160) | Refresh Rate : 60 Hertz | 178 Degree wide viewing angle. Connectivity: 3 HDMI ports to connect set top box, Blu Ray players, gaming console (HDMI 1 eARC supported) | 2 USB ports to connect hard drives and other USB devices | Dual-band Wi-Fi | Bluetooth 5.1. Sound : 102 Watts Output | JBL 6 Speakers System | Dolby Atmos for remarkable sound quality. Smart TV features: Google TV with Watchlist | Google Play Store | Google Assistant | Chromecast, Miracast, DLNA, Airplay | Auto Low Latency Mode for VRR | Supported Apps : Netflix, Youtube, Prime Video, Hotstar, SonyLiv, Hungama, JioCinema, Zee5, Eros Now. Display :10 bit Panel | Bezel-less Floating Display Design | ALLM | Decoding of Dolby Vision, HDR10, HLG | 1 Billion Colours | MEMC. Warranty Information: 2 Year Comprehensive Warranty on product provided by Hisense from date of purchase.. Installation: For requesting installation/wall mounting/demo of this product once delivered, please directly call Hisense Support for assistance (Please visit Hisense Website for Toll Free Numbers) and provide product's model name and seller's details mentioned on your invoice. The service center will allot you a convenient slot for the service. Easy Returns: This product is eligible for replacement within 10 days of delivery in case of any product defects, damage or features not matching the description.
Televisions

Recent Articles

Recent Quick Read

Recent Great People

1,059OFF
Butterfly Abs Hippo Tabletop Wet Grinder 2 Litre, 150 W, 230 V, Ac 50Hz, Cherry Red

Butterfly Abs Hippo Tabletop Wet Grinder 2 Litre, 150 W, 230 V, Ac 50Hz, Cherry Red

3,998 5,057
Wattage: 150 W; Voltage: 220-240V, 50-60Hz ; Revolution: 1300. No of Jars: 1; Jar Size: 2 Liters; Jar Material: Stainless Steel; Body Material: ABS body; Cord Material: 180 CM; Speed Control: 1 Speed Control. Read Manual Before Use. Powerful motor; Product Dimensions: 47.5 x 30.5 x 28 cm; Uniform grinding pressure and 4 Way grinding system. Shock proof ABS body, Feature: Detachable Stainless Steel drum & roller stones. Sturdy single phase high torque motor; Unbreakable and transparent lids made of polycarbonate material; Includes: Main Unit, 2 stones, lid, Warranty card with user manual.
Kitchen Appliances
802OFF
Bajaj RCX 1.8 DLX Rice Cooker, 1.8 Litre , White

Bajaj RCX 1.8 DLX Rice Cooker, 1.8 Litre , White

1,898 2,700
1.8-Litre capacity; 1Kg raw rice cooking capacity. The Cooking Bowl is Aluminum with anodized finish. Automatic cooking. Keep Warm : No. Warranty: 2 years on product. Power: 700 watts; Operating voltage: 230 volts.
Kitchen Appliances
4,171OFF
Amazon Brand - Solimo Xxxl Bean Bag Filled With Beans (Black And Brown)(Faux Leather)

Amazon Brand - Solimo Xxxl Bean Bag Filled With Beans (Black And Brown)(Faux Leather)

1,829 6,000
Dimensions: 71D x 71W x 124H Centimeters; Weight: 3600 Grams. Beans Required: No. Pattern: Faux Leather| Style: With Beans. Special Features: Velcro and zipper offer double protection to prevent beans from spilling out. Warranty: 1 year on manufacturing defects | Specific use: Bedroom, Lounge. Assembly Instruction: No assembly required.
Furniture
601OFF
amazon basics Vacuum Compression Storage Bags with Hand Pump - Medium, 5-Pack, Plastic

amazon basics Vacuum Compression Storage Bags with Hand Pump - Medium, 5-Pack, Plastic

599 1,200
Set of heavy-duty plastic vacuum compression storage bags. Compress bulky textiles, clothing, linens, and more at an 80% size reduction. Protects items from moisture, dirt, insects, mold, odors, and more. Hand pump included, or connect to any standard vacuum cleaner hose for faster results. Heavy-duty valve and double seal zipper for air tight protection. 5-pack of Medium bags.
Household
We Need Your Consent
By clicking “Accept Cookies”, you agree to the storing of cookies on your device to enhance your site navigation experience.
I Accept Cookies