Logging

Sapi-nt uses the Logback framework for logging. You can override the default Logback configuration by providing a logback.xml file in your application’s resources directory. They are also integrated with Sentry for system monitoring.

SPARQL Logs

Sapi-nt logs all SPARQL queries it performs by default, however the length of these log messages may be problematic for some integrations. You can compact these query logs by setting the sapi-nt.config.sparql.log.compact application property to true. Alternatively, you can disable them altogether by setting the sapi-nt.config.sparql.log.enabled property to false.

Format

Sapi-nt logs use the following format by default:

2020-07-23 17:31:13.614  INFO [-,a725b38141fd21df,a725b38141fd21df,false] 78590 --- [nio-8080-exec-1] c.e.sapint.filter.RequestLoggingFilter   : Request received: /api/doc/model

Property	Value
Timestamp	2020-07-23 17:31:13.614
Level	INFO
Process ID	78590
Thread	nio-8080-exec-1
Logger	com.epimorphics.sapint.filter.RequestLoggingFilter
Message	Request received: /api/doc/model
Sleuth Trace ID	a725b38141fd21df
Sleuth Span ID	a725b38141fd21df

JSON Format (Logstash)

We recommend emitting logs in JSON format by using the encoder by Logstash. In this case, you should add the following dependencies to your project:

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>6.6</version>
</dependency>
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.5.6</version>
</dependency>

And configure your logback.xml:

<!-- logback.xml -->
<configuration scan="true">
    <!-- Disable logback internal logs -->
    <statusListener class="ch.qos.logback.core.status.NopStatusListener" />
    <!-- Allows access to CONSOLE_LOG_PATTERN variable, which gives default logback pattern -->
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>

    <appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${CONSOLE_LOG_PATTERN}</pattern>
            <charset>utf8</charset>
        </encoder>
    </appender>

    <appender name="jsonstdout" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <excludeMdcKeyName>X-B3-SpanId</excludeMdcKeyName>
            <excludeMdcKeyName>X-B3-TraceId</excludeMdcKeyName>
            <excludeMdcKeyName>X-Span-Export</excludeMdcKeyName>
            <excludeMdcKeyName>spanExportable</excludeMdcKeyName>
            <excludeMdcKeyName>spanId</excludeMdcKeyName>
            <excludeMdcKeyName>traceId</excludeMdcKeyName>
            <timeZone>UTC</timeZone>
            <fieldNames>
                <timestamp>ts</timestamp>
                <version>version</version>
            </fieldNames>
            <throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
                <maxDepthPerThrowable>30</maxDepthPerThrowable>
                <maxLength>2048</maxLength>
                <shortenedClassNameLength>20</shortenedClassNameLength>
                <rootCauseFirst>true</rootCauseFirst>
            </throwableConverter>
        </encoder>
    </appender>

    <root level="info">
        <springProfile name="dev">
            <appender-ref ref="stdout" />
        </springProfile>
        <springProfile name="!dev">
            <appender-ref ref="jsonstdout" />
        </springProfile>
    </root>
</configuration>

See the Logstash documentation for more information about how to configure the encoder.

Message length limit

Sapi-nt provides a custom JsonProvider plugin that allows you to limit the size of, and truncate, messages. To enable this provider, add the following configuration to your Logstash encoder:

<encoder class="net.logstash.logback.encoder.LogstashEncoder">
    <fieldNames>
        <message>[ignore]</message>
    </fieldNames>
    <provider class="com.epimorphics.sapint.logging.MessageLengthJsonProvider">
        <maxLength>1000</maxLength>
        <trail>...</trail>
    </provider>
</encoder>

You can set the maximum acceptable message length with the maxLength tag, and optionally the trailing string used to denote truncated messages with trail.

JSON Format

Alternatively, you can configure your Sapi-nt application to emit logs in JSON format by overriding the logback.xml file and providing the necessary dependencies in your project’s pom.xml file. This can be useful for consuming the log output as structured data.

Example

<!-- pom.xml -->
<dependencies>
  <dependency>
    <groupId>ch.qos.logback.contrib</groupId>
    <artifactId>logback-json-classic</artifactId>
    <version>0.1.5</version>
  </dependency>
  <dependency>
    <groupId>ch.qos.logback.contrib</groupId>
    <artifactId>logback-jackson</artifactId>
    <version>0.1.5</version>
  </dependency>
</dependencies>

<!-- logback.xml -->
<configuration scan="true">
    <!-- Allows access to CONSOLE_LOG_PATTERN variable, which gives default logback pattern -->
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>

    <!-- Configure the Console appender for regular stdout logging -->
    <appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${CONSOLE_LOG_PATTERN}</pattern>
            <charset>utf8</charset>
        </encoder>
        <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
            <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
                <prettyPrint>false</prettyPrint>
            </jsonFormatter>
            <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
            <appendLineSeparator>true</appendLineSeparator>
        </layout>
    </appender>

    <!-- Configure the Sentry appender, overriding the logging threshold to the WARN level -->
    <appender name="Sentry" class="io.sentry.logback.SentryAppender">
        <!-- filter only allows logs with level warn or higher through to sentry -->
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>WARN</level>
        </filter>
    </appender>

    <!-- Enable the Console and Sentry appenders -->
    <root level="INFO">
        <appender-ref ref="Console" />
        <appender-ref ref="Sentry" />
    </root>
</configuration>