I wrote my own application for viewing structured logs

Not every project needs decentralized logging. In my case, it turned out to be easier to store logs in .json files of the format Compact Log Event Format (CLEF). I needed a simple and free solution for viewing logs.

Ready solution

At the time of writing, there was only 1 suitable application for reading logs in the CLEF format – Compact Log Viewer, available in the Microsoft Store.

Not bad at all, the application allowed you to view and filter logs. However, this decision had several disadvantages:

1. Each developer needs to install locally via Microsoft Store

2. You can only open 1 file at a time

3. You can't put it on a test server if it's Linux

4. Part of the UI takes up a useless graphic

5. After re-opening the application, the previously downloaded file was lost and had to be downloaded again

After a while, I got tired of constantly downloading logs from the server to my local machine to view them. Therefore, I decided to write my own application.

Requirements for the application being developed

I set the following requirements for the application being developed:

1. Show list of logs

2. Support viewing properties of individual lairs

3. Support filtering by properties

4. Support multiple file uploads

5. Save logs between sessions

6. Support running in Docker environment

Support for the Docker environment made it possible to make the application “available out of the box” by including it in the default docker-compose file on projects. Uploading to the server has also become possible.

Development process

To develop the project, there was a choice between React and Angular. Was chosen based on personal preference React + StyledComponents.

For UI components I chose a free solution RsuiteJs.

Flow applications:

1. Go to the log download page

2. Download logs

3. Redirect to log view page

Implementation of log loading

For forms I settled on the link Yup With Formik. Created wrappers over form components, FileUploader from RSuite didn't fit using the library ReactDropzone wrote a custom implementation.

Storing downloaded logs

To simplify the project, I abandoned the backend and NoSql database. I write logs to the browser storage using Localforage. Added AppState And AppStorage contexts.

//прослойка над хранилищем, контролирует согласованность данных за счет storageVersion
export interface AppStorage {
  loadedLogFilesInfo?: FileInfo[];
  csLogs?: CsLog[];
  storageVersion: number;
}
//Основной контекст приложения
export interface AppState {
  loadedLogFilesInfo?: FileInfo[];
  csLogs?: CsLog[];
}

export function BaseLayout(): ReactElement {
  //кастомная реализация светлой и темной темы 
  const theme = useTheme();

  return (
    <ErrorBoundary FallbackComponent={ErrorFallback}>
      <AppStorageContextProvider>
        <AppStateContextProvider>
          <RSuiteProvider theme={theme}>
            <AppLayout />
          </RSuiteProvider>
        </AppStateContextProvider>
      </AppStorageContextProvider>
    </ErrorBoundary>
  );
}

To further display logs in the list, you need to set what is to be taken as the unique Id of each entry in the file. Consider the example logs below. It can be seen that there is no id field. You can try using "@t"however, the log time does not provide a guarantee of uniqueness.

{"@t":"2024-03-06T07:43:20.5672449Z","@mt":"Starting up","@l":"Information","EnvironmentName":"Staging"}
{"@t":"2024-03-06T07:43:23.3405140Z","@mt":"Executing ViewResult, running view {ViewName}.","@l":"Warning","@tr":"f3e14fda062a0ae0d641782c77ee0617","@sp":"0aee0c652548be62","ViewName":"Index","EventId":{"Id":1,"Name":"ViewResultExecuting"},"SourceContext":"Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor","ActionId":"9ea02b27-cd15-4432-82b1-7f9768a48e4b","ActionName":"DealerServiceSystem.Web.Controllers.CheckListCategoryController.Index (DealerServiceSystem.Web)","RequestId":"40000bc6-0002-f200-b63f-84710c7967bb","EnvironmentName":"Staging"}
{"@t":"2024-03-06T07:43:23.7431244Z","@mt":"Executed ViewResult - view {ViewName} executed in {ElapsedMilliseconds}ms.","@l":"Debug","@tr":"f3e14fda062a0ae0d641782c77ee0617","@sp":"0aee0c652548be62","ViewName":"Index","ElapsedMilliseconds":404.2372,"EventId":{"Id":4,"Name":"ViewResultExecuted"},"SourceContext":"Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor","ActionId":"9ea02b27-cd15-4432-82b1-7f9768a48e4b","RequestId":"40000bc6-0002-f200-b63f-84710c7967bb","EnvironmentName":"Staging"}
{"@t":"2024-03-06T07:43:23.7466340Z","@mt":"Executed action in {ElapsedMilliseconds}ms","@l":"Warning","@tr":"f3e14fda062a0ae0d641782c77ee0617","@sp":"0aee0c652548be62","ElapsedMilliseconds":536.3229,"EventId":{"Id":105,"Name":"ActionExecuted"},"SourceContext":"Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker","RequestId":"40000bc6-0002-f200-b63f-84710c7967bb","EnvironmentName":"Staging"}

I solved this problem by adding “artificial” idgenerated via crypto-randomuidthe following structures came out:

export interface CsLog {
  id: string;
  data: CsLogData;
}

export interface CsLogData {
  [key: string]: any;
  '@t': Date;
  '@mt': string;
  '@l'?: 'Verbose' | 'Debug' | 'Information' | 'Warning' | 'Error' | 'Fatal';
}

Implementation of viewing and filtering logs

Implemented pagination in a separate component, used as follows:

const [paginatedLogs, setPaginatedLogs] = useState<CsLog[] | null>(null)

<StyledPagination values={filteredLogs} onChange={setPaginatedLogs} />

To filter by logs I used SearchJssyntax of the simplest filter { "data.<property_name>": "value" }, implementation:

searchjs.matchArray(appState.csLogs, JSON.parse(filter))

I did the filter validation straight away, I’m trying to parse json, the error is incorrect format:

export const logsViewPageFormSchema: yup.Schema<LogsViewPageFormData> = yup.object({
  filter: yup.string().test('json', 'Filter must have JSON format', (value) => {
    if (value === undefined) {
      return true;
    }

    try {
      JSON.parse(value);

      return true;
    } catch (error) {
      return false;
    }
  })
});

The result was the following page:

Publishing an application

To make the project publicly available, I posted its code in a public repository on GitHub by this link.

Set up a workflow for the Docker Image build and publication in the DockerHub repository link. The application can be launched using the following docker-compose file, the project will be available on http://localhost:8080 port:

version: "3.8"
services:
  client:
    image: "migiki/cs-logs-viewer:latest"
    container_name: cs-logs-viewer
    ports:
      - "8080:80"

Also added MkDocs documentation for the project, available at link

Results

Developed a React CS Logs viewer for viewing structured logs, the image of which weighs about 55mbwhile the analogue considered above occupies 300MB. The application can be deployed to a test server or supplied in a docker-compose file along with your project code, so that you can view the logs on any system with a Docker environment installed.

I will be glad to see your suggestions and comments!