🍃 Tellia / 🥷🏿 Engineering

Appearance

Observability & Logging Guidelines ​

Welcome to the Agri Backend observability guide! 🚀

Our application logs and metrics are sent to Elasticsearch and visualized in Kibana. We also use Elastic APM for distributed tracing and request correlation.

When working locally your logs & metrics can also be searchable. Sets the relevant environment variables and NODE_ENV=local.

Logging with NestJS ​

  • Use the custom logger (MyLogger) in your NestJS services/components:
typescript
@Injectable()
export class SomeService {
  private readonly logger: MyLogger = new MyLogger(SomeService.name);
  constructor() {}

  async doSomething(input: any) {
    const result = await this.compute(input);
    this.logger.log(
      {
        savedTimeMs: result,
        input: {
          questionsCpt: questionDocs.length,
          charactersCpt: characterDocs.length,
        },
      },
      `Time saved computed: ${JSON.stringify(result)}`,
    );
  }
}
  • Context attributes (the first argument to log) are indexed and searchable in Elasticsearch/Kibana.
  • The log message (second argument) is indexed as a string.

Using logger ​

  • Always inject MyLogger via the constructor:
typescript
private readonly logger = new MyLogger(Service.name)
constructor() {}
  • Add relevant context to your logs for better searchability and debugging.
  • Don't be crazy with context. ES is a paid service even if it can handle a lot we should try to avoid putting huge data there

Adding log context / searchable fields ​

Log context allows you to add structured metadata to your logs, making them more searchable and informative in the monitoring dashboard.

How to Add Context ​

When logging, you can pass an optional context object as the second argument. This object's attributes become indexed and searchable in Elasticsearch/Kibana.

typescript
// Basic logging with context
this.logger.log('User logged in', {
  userId: user.id,
  email: user.email,
  loginMethod: 'email',
  ipAddress: req.ip,
});

// Logging with error context
this.logger.error('Database connection failed', {
  errorCode: 'DB_CONN_001',
  connectionString: 'mongodb://...',
  retryAttempt: 3,
});

Best Practices for Log Context ​

  1. Use Primitive Types: Stick to simple types like strings, numbers, and booleans
  2. Keep it Concise: Don't log massive objects or sensitive information
  3. Be Consistent: Use similar key names across different log statements
  4. Include Relevant Details: Add context that helps in debugging or understanding the system state

Searchable Context Examples ​

  • User-related logs: userId, email, role
  • Performance metrics: executionTimeMs, resourceUsed
  • System events: serviceVersion, environmentName
  • Error tracking: errorCode, stackTrace

What to Avoid ​

  • Avoid logging sensitive data (passwords, tokens)
  • Don't log entire large objects
  • Keep the context object relatively small and meaningful
typescript
// ✅ Good: Concise, relevant context
this.logger.log('Processing user request', {
  requestId: uuid(),
  endpoint: '/api/users',
});

// ❌ Bad: Too much information, potential security risk
this.logger.log('User details', {
  user: JSON.stringify(userObject), // Avoid logging entire objects
  sensitiveData: user.password, // Never log sensitive information
});

Performance Considerations ​

While Elasticsearch can handle a lot of data, be mindful of the volume and complexity of your log contexts to manage costs and performance.