Store Drupal logs on Amazon S3

On24th Jun 2019, 2022-12-01T08:00:00+05:30 Read Pause Resume Stop
Store Drupal logs on Amazon S3

Store Drupal logs on Amazon S3 via hook_watchdog, so that you can get rid of heavy logs on your drupal database and can later read from the S3.

For this todo on Drupal

  • You should use the "hook_watchdog" hook, where this hook allows modules to route log events to custom destinations.
  • In our case, our custom destination will be S3.
  • Initially we need to get the access to AWS and appropriate S3 bucket.
  • Store the connection configuration details of S3 buckets in the Drupal.
  • In the hook_watchdog, read the S3 Auth configs.
  • Create a connection to AWS S3, on success it return back with the S3 object.
  • Check if any buckets exist to write the logs, of not you should create bucket on S3 for logs.
  • Next, Create a log directory and log file, populate it with the data came on hook_watchdog.
  • Next, Write to the S3 bucket with parameters which "putObject" expects.
  • Next, Remove the log directory & file which is created in the process.
  • Check the S3 bucket via S3 UI, you could see the logs

Here's the piece of code, via we log to S3.

watchdog(    'module_2',    t('Info: Successfully completed.'),    array($data),    WATCHDOG_INFO);watchdog(    'module_3',    t('Error: Issue while processing.'),    array($exception, $data),    WATCHDOG_ERROR);

Here, the piece of code which helps to connect & write log's to S3.

<?phpuse \Aws\S3\S3Client;function audit_watchdog(array $log_entry) {    // Set the modules which needs to be logged to S3    $modules = array(        'module_1',        'module_2',        'module_3',    );    if (in_array($log_entry['type'], $modules)) {        if(isset($log_entry) && !empty($log_entry)) {            // Get the S3 config details            $s3_config = variable_get("s3_config");            $s3_bucket = $s3_config->s3_bucket;            $s3_region = $s3_config->s3_region;            $s3_key = $s3_config->s3_key;            $s3_secret = $s3_config->s3_secret;            try {                if (!empty($s3_bucket) && !empty($s3_region) && !empty($s3_key) && !empty($s3_secret)) {                    // Create AWS connection                    $s3 = create_aws_connection($s3_region, $s3_key, $s3_secret);                    if (!empty($s3) && is_object($s3) && $s3 != FALSE) {                        if ($s3->doesBucketExist($s3_bucket)) {                            $log_path = 'public://logs/to_aws';                            // Create log directory                            $dir_path = create_aws_log_directory($log_path);                            // Create log file                            $file_path = create_aws_log_file($log_entry, $dir_path);                            // Get the name of log file                            $keyname = get_keyname_for_log_file($log_entry['severity'], $file_path);                            // Store the log file to S3                            $s3->putObject([                                'Bucket' => $s3_bucket,                                'Key' => $keyname,                                'Body' => '',                                'SourceFile' => $file_path,                            ]);                            // Remove the directory & file created                            if (is_dir($dir_path)) {                                rmdir_recursive($dir_path);                                rmdir_recursive('public://logs');                            }                        }                    }                }            }            catch (Exception $e) {                // Exception is ignored so that watchdog does not break pages during the                // installation process or is not able to create the watchdog table during                // installation.            }        }    }}function create_aws_connection($s3_region, $s3_key, $s3_secret) {    $s3 = new S3Client([        'version' => 'latest',        'region'  => $s3_region,        'credentials' => [            'key'    => $s3_key,            'secret' => $s3_secret,        ]    ]);    $buckets = $s3->listBuckets()->get('Buckets');    if (isset($buckets) && !empty($buckets)) {        return $s3;    }    else {        return FALSE;    }}function create_aws_log_directory($log_path) {    if (!is_dir($log_path)) {        mkdir($log_path, 0777, true);        chmod($log_path, 0777);    }    return $log_path;}function create_aws_log_file($log_entry, $dir_path) {    $content = json_encode($log_entry);    $log_file_name = $log_entry['type'] . '-' . date("Y-m-d-H-i-s") . '-' . preg_replace("/^.*\./i","", microtime(true)) . '.log';    $file_path = $dir_path . '/' . $log_file_name;    $log_file = fopen($file_path, "w");    $write_log_file = fwrite($log_file, $content);    $close_log_file = fclose($log_file);    $chmod_log_file = chmod($file_path, 0777);    return $file_path;}function get_keyname_for_log_file($severity, $file_path) {    $watchdog_array = array(        "0" => "WATCHDOG_ERROR",        "1" => "WATCHDOG_INFO",    );    return 'drupal-logs/' . $watchdog_array[$severity] . '/' . basename($file_path);}function rmdir_recursive($dir) {    foreach(scandir($dir) as $file) {        if ('.' === $file || '..' === $file) continue;        if (is_dir("$dir/$file")) rmdir_recursive("$dir/$file");        else unlink("$dir/$file");    }    rmdir($dir);}

Advantage of having Logs on S3

  • Reduce the number of DB log entries on Drupal database.
  • Completely Keep the Audit system outside Drupal, So Prod instance will play smooth.

Cheers :)

5,000OFF
Samsung Galaxy S24 Ultra 5G AI Smartphone (Titanium Gray, 12GB, 512GB Storage)

Samsung Galaxy S24 Ultra 5G AI Smartphone (Titanium Gray, 12GB, 512GB Storage)

1,39,999 1,44,999
Brand Samsung
Model Name Samsung Galaxy S24 Ultra 5G
Network Service Provider Unlocked for All Carriers
Operating System Android 14.0
Cellular Technology 5G

About this item

  • Meet Galaxy S24 Ultra, the ultimate form of Galaxy Ultra with a new titanium exterior and a 17.25cm (6.8") flat display. It's an absolute marvel of design.
  • The legacy of Galaxy Note is alive and well. Write, tap and navigate with precision your fingers wish they had on the new, flat display.
  • With the most megapixels on a smartphone and AI processing, Galaxy S24 Ultra sets the industry standard for image quality every time you hit the shutter. What's more, the new ProVisual engine recognizes objects — improving colour tone, reducing noise and bringing out detail.
  • A new way to search is here with Circle to Search. While scrolling your fav social network, use your S Pen or finger to circle something and get Google Search results.
  • Victory can be yours with the new Snapdragon 8 Gen 3 for Galaxy. Faster processing gives you the power you need for all the gameplay you want. Then, manifest graphic effects in real time with ray tracing for hyper-realistic shadows and reflections.
Mobiles & Accessories
3,500OFF
Redmi 13C 5G (Startrail Green, 4GB RAM, 128GB Storage) | MediaTek Dimensity 6100+ 5G | 90Hz Display

Redmi 13C 5G (Startrail Green, 4GB RAM, 128GB Storage) | MediaTek Dimensity 6100+ 5G | 90Hz Display

10,499 13,999
Brand Redmi
Model Name Redmi 13C 5G
Network Service Provider Unlocked for All Carriers
Operating System MIUI 14, Android 13.0
Cellular Technology 5G

About this item

  • Processor: Powerful MediaTek Dimensity 6100+ 5G SoC | 8GB of RAM including 4GB virtual | 6.74" HD+ 90Hz display with Corning Gorilla Glass 3 Protection | 50MP AI Dual camera |Fast Side fingerprint | 5000mAh Battery
  • 6.74" HD+ 90Hz display with Corning Gorilla Glass 3 Protection, 600nits in High Brightness mode
  • 50MP AI Dual camera with Primary sensor of f/1.8 (4-in-1 super pixel) with the following modes: Photo | Portrait | Night | Video | 50MP mode | Time-lapse | Classic film filters | Frame | HDR | Google lens | Voice Shutter
Mobiles & Accessories
11,401OFF
Apple iPhone 13 (128GB) - Midnight

Apple iPhone 13 (128GB) - Midnight

48,499 59,900
Brand Apple
Model Name iPhone
Network Service Provider Unlocked for All Carriers
Operating System iOS 14
Cellular Technology 5G

About this item

  • 15 cm (6.1-inch) Super Retina XDR display
  • Cinematic mode adds shallow depth of field and shifts focus automatically in your videos
  • Advanced dual-camera system with 12MP Wide and Ultra Wide cameras; Photographic Styles, Smart HDR 4, Night mode, 4K Dolby Vision HDR recording
  • 12MP TrueDepth front camera with Night mode, 4K Dolby Vision HDR recording
  • A15 Bionic chip for lightning-fast performance
Mobiles & Accessories

Related Articles

13,000OFF
LG 8 Kg 5 Star Inverter Direct Drive Touch Panel Fully Automatic Front Load Washing Machine (FHM1408BDM, Steam for Hygiene, In-Built Heater, 6 Motion DD, Middle Black)

LG 8 Kg 5 Star Inverter Direct Drive Touch Panel Fully Automatic Front Load Washing Machine (FHM1408BDM, Steam for Hygiene, In-Built Heater, 6 Motion DD, Middle Black)

34,990 47,990
Capacity 8 Kilograms
Colour Middle Black
Brand LG
Product Dimensions 60D x 55W x 85H Centimeters
Special Feature Inverter, 6 Motion Direct Drive, Child Lock, Auto Restart, Hygiene Steam, Time Remaining Display, Inbuilt HeaterInverter, 6 Motion Direct Drive, Child Lock, Auto Restart, Hygiene Steam, Time Remaining Display, Inbuilt Heater
Cycle Options Active Steam
Voltage 230 Volts
Controls Type Knob
Maximum Rotational Speed 1400 RPM
Access Location Front Load

About this item

  • Fully-automatic front load washing machine with Hygiene Steam/direct-drive technology: Best Wash Quality, Energy and Water efficient
  • Capacity 8 kg: Suitable for large families
  • Energy Star rating: 5 Star best in class efficiency; Energy consumption – 0.06 KWh/kg/cycle & Water Consumption: 6.6 L/Kg/Cycle (Please refer BEE label for more information)
  • Manufacturer Warranty: 2 Years Comprehensive & 10 Years on Motor T&C
  • 1400 RPM: higher spin speeds helps in faster drying
  • Wash Programs: 10 washing programs - Cotton: designed for normally soiled cotton clothes | Cotton Large - provides optimized washing performance for large amounts of laundry | Mix - cleans a variety of different fabrics at the same time | Easy Care –for polyamide, acrylic and polyester fabrics | Baby Care – eliminates allergen | Sportswear - suitable for sportswear such as jogging clothes | Delicate – suitable for undergarments and delicate fabrics | Wool - suitable for lacy knickers or that silk blouse | Quick 30 - Get express wash for lightly soiled clothes, all in 30 mins* | Rinse + Spin - ideal for cloth diapers or heavily soiled items
  • Drum / Pulsator type: Fully Stainless Steel Drum built to bring convenience and durability, this stainless steel drum makes your washing machine last longer. Not just this, its stainless steel lifter keeps the insides of the washing machine hygienic
Home & Kitchen

Recent Articles

Recent Quick Read

Recent Great People

1,840OFF
Amazon Basics 8-Piece Non-Stick Cookware Set| 2-Way 3 Layer Non-Stick Coating | PFOA Free | High Temperature Resistant Exterior Coating, Black,Aluminium

Amazon Basics 8-Piece Non-Stick Cookware Set| 2-Way 3 Layer Non-Stick Coating | PFOA Free | High Temperature Resistant Exterior Coating, Black,Aluminium

2,859 4,699
  • Enjoy healthy and efficiently cooked food made using this non-stick 8-piece cookware set
  • The cookware set is made from pressed aluminium that is ultra-conductive, corrosion-free; it ensures quick heating and helps save energy
  • The heat-resistant phenolic handles remain cool even at temperatures when cooking the food
  • The coatings are 100% free from PFOA, lead, cadmium, nickel, and arsenic and approved by USFDA and European Food Safety Authority (EFSA)
  • The 2-layered exterior coating is scratch-resistant while the 3-layered interior is coated with a water-based eco-friendly PFOA-free coating
  • The durable and lightweight cookware is suitable for meeting the needs of daily cooking
  • The cookware is gas-stove compatible and dishwasher-safe
  • Package contents: Fry pan - 20 cm, fry pan - 26 cm, sauce pan - 16 cm with glass lid, sauce pan - 18 cm with glass lid, casserole - 20 cm with glass lid
Home & Kitchen
540OFF
Amazon Brand - Solimo Non-Stick Kadai with Glass Lid | Granite Finish | Induction Base | PFOA Free | High Temperature Resistant Exterior Coating | 22 cm | Grey

Amazon Brand - Solimo Non-Stick Kadai with Glass Lid | Granite Finish | Induction Base | PFOA Free | High Temperature Resistant Exterior Coating | 22 cm | Grey

559 1,099
  • HEAT CONDUCTIVITY: Made with pressed aluminium, the tawa provides high conductivity which enables quick heat distribution and even cooking
  • PFOA-FREE: Being PFOA-free, the product is safe for daily use
  • NON-STICK: The non-stick feature makes it super convenient to use as food doesn't stick to the cookware
  • PHENOLIC HANDLES: This product has phenolic handles which remain cool to the touch during cooking
  • DISHWASHER-SAFE: The product is dishwasher-safe and hence, easy to clean and maintain
  • PACKAGE CONTENTS: 22cm Kadai, 1 Glass Lid
Home & Kitchen
2,099OFF
London Hills Women's Casual Printed Round Neck, Oversized Longline Drop Shoulder Boho Style Regular Fit T-Shirt Pack Of 2

London Hills Women's Casual Printed Round Neck, Oversized Longline Drop Shoulder Boho Style Regular Fit T-Shirt Pack Of 2

499 2,598
  • Fit Type: Oversized Fit
  • Fabric: Cotton blend
  • Sleeve Type: Half Sleeve
  • Neck Style: Round Neck
  • Pattern : Printed
Clothing & Accessories
251OFF
Amazon Brand - Solimo Stainless Steel Induction Bottom Kadhai (25cm), Silver

Amazon Brand - Solimo Stainless Steel Induction Bottom Kadhai (25cm), Silver

449 700
  • A safe and robust cooking utensil made from 100% food grade Stainless Steel.
  • Product Dimensions : 25CM diameter , depth : 7.8CM.
  • Serves various cooking, frying and serving options for daily kitchen needs.
  • Made with high-quality induction bottom that is also gas stove compatible.
  • Sturdy and durable with a thickness of 0.6mm and capacity of 3000ml.
  • A stable grip with double riveted ear handles. Easy to clean in a dishwasher.
Kitchen & Dining
We Need Your Consent
By clicking “Accept Cookies”, you agree to the storing of cookies on your device to enhance your site navigation experience.
I Accept Cookies