Feature request: Always log message even when sampling is enabled

[bestickley]

Use case

Important

See this comment for an updated scope & context on the issue.

I'd like to use Lambda Powertools Logger to log audit information. This info needs to be logged 100% of the time whereas my other log information doesn't. When I enable a sample rate, I'd like to be able to be sure the audit information is logged.

Solution/User Experience

const logger = new Logger({
  serviceName: "MyService",
});
// don't care about "always", could be whatever you think makes sense
logger.always.info("Here is my important log information");

Alternative solutions

No response

Acknowledgment

• This feature request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Python, Java

[dreamorosi]

Hi @bestickley thank you for opening this issue.

Please allow us some time to review this and come up with an answer.

[dreamorosi]

At the moment we don't support this type of behavior, however I think it's worth discussing and I would like also to hear the opinion of other maintainers and customers.

In the meantime, I would like to suggest an alternative to achieve a similar behavior. The alternative revolves around using two logger instances:

import { Logger, injectLambdaContext } from "@aws-lambda-powertools/logger";
import middy from "@middy/core";

const logger = new Logger({
  serviceName: "MyService",
  sampleRateValue: 0.01,
});
const auditLogger = logger.createChild({
  sampleRateValue: 1,
});

export const handler = middy(async () => {
  logger.info("I log sometimes");
  auditLogger.info("I always log");
})
  .use(injectLambdaContext(logger))
  .use(injectLambdaContext(auditLogger));

These should produce the following logs:

{
    "cold_start": false,
    "function_arn": "arn:aws:lambda:eu-west-1:536254204126:function:lambda-powertools-playground-fn",
    "function_memory_size": 128,
    "function_name": "lambda-powertools-playground-fn",
    "function_request_id": "6bb4c509-31d0-4dc7-9e16-f08f22a70b59",
    "level": "INFO",
    "message": "I log sometimes",
    "sampling_rate": 0.01,
    "service": "MyService",
    "timestamp": "2022-11-13T16:47:24.460Z",
    "xray_trace_id": "1-63711f9c-4c1fc3284f97466b7ff88473"
}
{
    "cold_start": false,
    "function_arn": "arn:aws:lambda:eu-west-1:536254204126:function:lambda-powertools-playground-fn",
    "function_memory_size": 128,
    "function_name": "lambda-powertools-playground-fn",
    "function_request_id": "6bb4c509-31d0-4dc7-9e16-f08f22a70b59",
    "level": "INFO",
    "message": "I always log",
    "sampling_rate": 1,
    "service": "service_undefined",
    "timestamp": "2022-11-13T16:47:24.461Z",
    "xray_trace_id": "1-63711f9c-4c1fc3284f97466b7ff88473"
}

Notice the usage of createChild & the fact that I'm using the injectLambdaContext middleware twice (however it would work also with decorator & manual modes using the corresponding methods).

---

I'm going to add the need-customer-feedback label to hear also from other customers. If anyone comes across this issue and finds it interesting, please add a 👍 and consider sharing your use case.

[saragerion]

Agree with Andrea. I would also like to add that to be even extra sure that all the audit logs will be logged, you can set a custom logLevel in the constructor that will override any level set via env variables:

const auditLogger = logger.createChild({
  sampleRateValue: 1,
  logLevel: 'DEBUG' 
});

This is a very valid and common use case and we may consider adding a dedicated section in the docs.

---

Edit

Example:

import { Logger } from "@aws-lambda-powertools/logger";

const logger = new Logger({
  serviceName: "MyService",
  logLevel: 'ERROR',
  sampleRateValue: 0.5,
});

const auditLogger = logger.createChild({
  sampleRateValue: 1,
  logLevel: 'DEBUG'
});

export const handler = async () => {
  
  logger.error("I always log");
  logger.info("I log 50% of the time");
  logger.info("I log 50% of the time");
  logger.info("I log 50% of the time");
  
  auditLogger.debug("I always log");
  
};

[nateiler]

Weighing in on this potential feature as well. Adding a child logger is an interesting approach (and works for now), however it doesn't excite me to have all the log levels available to the 'audit' logger... feels a bit like a hack.

In an ideal world, we could do something like logger.audit( ... ). Additionally, having a persistent attribute that we could filter on within CloudWatch is helpful.

At the moment, we're doing something like this:

const logger = new Logger({ logLevel: 'WARN' });
const audit = logger.createChild({
  logLevel: "INFO",
  persistentLogAttributes: { audit: true },
});

We're only logging with audit.info() as to not disturb WARN and ERROR metrics and in CloudWatch we can filter by the persistent 'audit' attribute.