Creating a DSL for generating images

Hello, Habr! A few days remain until the launch of a new course from OTUS “Backend development with Kotlin”… On the eve of the start of the course, we have prepared for you the translation of another interesting material.


Often when solving problems related to computer vision, the lack of data becomes a big problem. This is especially true when working with neural networks.

How great would it be if we had a limitless source of new original data?

This thought prompted me to develop a Domain Specific Language that allows images to be created in various configurations. These images can be used to train and test machine learning models. As the name suggests, generated DSL images can usually only be used in a narrowly focused area.

Language requirements

In my particular case, I need to focus on object detection. The language compiler must generate images that meet the following criteria:

  • images contain different forms (for example, emoticons);
  • the number and position of individual figures is customizable;
  • image size and shapes are customizable.

The language itself should be as simple as possible. I want to determine the size of the output image first and then the size of the shapes. Then I want to express the actual configuration of the image. To keep things simple, I think of the image as a table, where each shape fits into a cell. Each new row is filled with forms from left to right.

Implementation

To create a DSL, I chose the combination ANTLR, Kotlin and GradleANTLR is a parser generator. Kotlin Is a JVM language similar to Scala. Gradle is a build system similar to sbt

Necessary environment

To complete the steps described, you need Java 1.8 and Gradle 4.6.

Initial setup

Create a folder to contain the DSL.

> mkdir shaperdsl
> cd shaperdsl

Create a file build.gradle… This file is needed to list project dependencies and configure additional Gradle tasks. If you want to reuse this file, you only need to change the namespaces and the main class.

> touch build.gradle

Below is the content of the file:

buildscript {
   ext.kotlin_version = '1.2.21'
   ext.antlr_version = '4.7.1'
   ext.slf4j_version = '1.7.25'

   repositories {
     mavenCentral()
     maven {
        name 'JFrog OSS snapshot repo'
        url  'https://oss.jfrog.org/oss-snapshot-local/'
     }
     jcenter()
   }

   dependencies {
     classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
     classpath 'com.github.jengelman.gradle.plugins:shadow:2.0.1'
   }
}

apply plugin: 'kotlin'
apply plugin: 'java'
apply plugin: 'antlr'
apply plugin: 'com.github.johnrengelman.shadow'

repositories {
  mavenLocal()
  mavenCentral()
  jcenter()
}

dependencies {
  antlr "org.antlr:antlr4:$antlr_version"
  compile "org.antlr:antlr4-runtime:$antlr_version"
  compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
  compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
  compile "org.apache.commons:commons-io:1.3.2"
  compile "org.slf4j:slf4j-api:$slf4j_version"
  compile "org.slf4j:slf4j-simple:$slf4j_version"
  compile "com.audienceproject:simple-arguments_2.12:1.0.1"
}

generateGrammarSource {
    maxHeapSize = "64m"
    arguments += ['-package', 'com.example.shaperdsl']
    outputDirectory = new File("build/generated-src/antlr/main/com/example/shaperdsl".toString())
}
compileJava.dependsOn generateGrammarSource

jar {
    manifest {
        attributes "Main-Class": "com.example.shaperdsl.compiler.Shaper2Image"
    }

    from {
        configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
    }
}

task customFatJar(type: Jar) {
    manifest {
        attributes 'Main-Class': 'com.example.shaperdsl.compiler.Shaper2Image'
    }
    baseName="shaperdsl"
    from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
    with jar
}

Language parser

Parser is built like grammar ANTLR

mkdir -p src/main/antlr
touch src/main/antlr/ShaperDSL.g4

with the following content:

grammar ShaperDSL;

shaper      : 'img_dim:' img_dim ',shp_dim:' shp_dim '>>>' ( row ROW_SEP)* row '<<<' NEWLINE* EOF;
row       : ( shape COL_SEP )* shape ;
shape     : 'square' | 'circle' | 'triangle';
img_dim   : NUM ;
shp_dim   : NUM ;

NUM       : [1-9]+ [0-9]* ;
ROW_SEP   : '|' ;
COL_SEP   : ',' ;

NEWLINE   : 'rn' | 'r' | 'n';

Now you can see how the structure of the language becomes clearer. To generate the grammar source code run:

> gradle generateGrammarSource

As a result, you will get the generated code in build/generate-src/antlr...

> ls build/generated-src/antlr/main/com/example/shaperdsl/
ShaperDSL.interp  ShaperDSL.tokens  ShaperDSLBaseListener.java  ShaperDSLLexer.interp  ShaperDSLLexer.java  ShaperDSLLexer.tokens  ShaperDSLListener.java  ShaperDSLParser.java

Abstract syntax tree

The parser converts the source code into an object tree. The object tree is what the compiler uses as a data source. To get the AST, you first need to define the tree metamodel.

> mkdir -p src/main/kotlin/com/example/shaperdsl/ast
> touch src/main/kotlin/com/example/shaper/ast/MetaModel.kt

MetaModel.kt contains the definitions of the object classes used in the language, starting at the root. They all inherit from the interface Node... The tree hierarchy is visible in the class definition.

package com.example.shaperdsl.ast

interface Node

data class Shaper(val img_dim: Int, val shp_dim: Int, val rows: List): Node

data class Row(val shapes: List): Node

data class Shape(val type: String): Node

Next, you need to match the class with ASD:

> touch src/main/kotlin/com/example/shaper/ast/Mapping.kt

Mapping.kt used to build ASD using the classes defined in MetaModel.ktusing data from the parser.

package com.example.shaperdsl.ast

import com.example.shaperdsl.ShaperDSLParser

fun ShaperDSLParser.ShaperContext.toAst(): Shaper = Shaper(this.img_dim().text.toInt(), this.shp_dim().text.toInt(), this.row().map { it.toAst() })

fun ShaperDSLParser.RowContext.toAst(): Row = Row(this.shape().map { it.toAst() })

fun ShaperDSLParser.ShapeContext.toAst(): Shape = Shape(text)

The code on our DSL:

img_dim:100,shp_dim:8>>>square,square|circle|triangle,circle,square<<<

Will be converted to the following ASD:

Compiler

The compiler is the last part. He uses ASD to obtain a specific result, in this case, an image.

> mkdir -p src/main/kotlin/com/example/shaperdsl/compiler
> touch src/main/kotlin/com/example/shaper/compiler/Shaper2Image.kt

There is a lot of code in this file. I will try to clarify the main points.

ShaperParserFacade Is a shell on top ShaperAntlrParserFacadewhich creates the actual AST from the provided source code.

Shaper2Image is the main compiler class. After it receives the AST from the parser, it goes through all the objects inside it and creates graphic objects, which it then inserts into the image. Then it returns the binary representation of the image. There is also a function main in the companion object of the class, allowing testing.

package com.example.shaperdsl.compiler

import com.audienceproject.util.cli.Arguments
import com.example.shaperdsl.ShaperDSLLexer
import com.example.shaperdsl.ShaperDSLParser
import com.example.shaperdsl.ast.Shaper
import com.example.shaperdsl.ast.toAst
import org.antlr.v4.runtime.CharStreams
import org.antlr.v4.runtime.CommonTokenStream
import org.antlr.v4.runtime.TokenStream
import java.awt.Color
import java.awt.image.BufferedImage
import java.io.ByteArrayInputStream
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.InputStream
import javax.imageio.ImageIO

object ShaperParserFacade {

    fun parse(inputStream: InputStream) : Shaper {
        val lexer = ShaperDSLLexer(CharStreams.fromStream(inputStream))
        val parser = ShaperDSLParser(CommonTokenStream(lexer) as TokenStream)
        val antlrParsingResult = parser.shaper()
        return antlrParsingResult.toAst()
    }

}


class Shaper2Image {

    fun compile(input: InputStream): ByteArray {
        val root = ShaperParserFacade.parse(input)
        val img_dim = root.img_dim
        val shp_dim = root.shp_dim

        val bufferedImage = BufferedImage(img_dim, img_dim, BufferedImage.TYPE_INT_RGB)
        val g2d = bufferedImage.createGraphics()
        g2d.color = Color.white
        g2d.fillRect(0, 0, img_dim, img_dim)

        g2d.color = Color.black
        var j = 0
        root.rows.forEach{
            var i = 0
            it.shapes.forEach {
                when(it.type) {
                    "square" -> {
                        g2d.fillRect(i * (shp_dim + 1), j * (shp_dim + 1), shp_dim, shp_dim)
                    }
                    "circle" -> {
                        g2d.fillOval(i * (shp_dim + 1), j * (shp_dim + 1), shp_dim, shp_dim)
                    }
                    "triangle" -> {
                        val x = intArrayOf(i * (shp_dim + 1), i * (shp_dim + 1) + shp_dim / 2, i * (shp_dim + 1) + shp_dim)
                        val y = intArrayOf(j * (shp_dim + 1) + shp_dim, j * (shp_dim + 1), j * (shp_dim + 1) + shp_dim)
                        g2d.fillPolygon(x, y, 3)
                    }
                }
                i++
            }
            j++
        }

        g2d.dispose()
        val baos = ByteArrayOutputStream()
        ImageIO.write(bufferedImage, "png", baos)
        baos.flush()
        val imageInByte = baos.toByteArray()
        baos.close()
        return imageInByte

    }

    companion object {

        @JvmStatic
        fun main(args: Array) {
            val arguments = Arguments(args)
            val code = ByteArrayInputStream(arguments.arguments()["source-code"].get().get().toByteArray())
            val res = Shaper2Image().compile(code)
            val img = ImageIO.read(ByteArrayInputStream(res))
            val outputfile = File(arguments.arguments()["out-filename"].get().get())
            ImageIO.write(img, "png", outputfile)
        }
    }
}

Now that everything is ready, let's build the project and get a jar file with all dependencies (uber jar).

> gradle shadowJar
> ls build/libs
shaper-dsl-all.jar

Testing

All we have to do is check if everything works, so try entering this code:

> java -cp build/libs/shaper-dsl-all.jar com.example.shaperdsl.compiler.Shaper2Image 
--source-code "img_dim:100,shp_dim:8>>>circle,square,square,triangle,triangle|triangle,circle|square,circle,triangle,square|circle,circle,circle|triangle<<<" 
--out-filename test.png

A file will be created:

.png

which will look like this:

Conclusion

It is a simple DSL, it is not secure, and will probably break if misused. However, it suits my purpose well, and I can use it to create any number of unique image samples. It can be easily extended for more flexibility and can be used as a template for other DSLs.

A complete DSL example can be found in my GitHub repository: github.com/cosmincatalin/shaper...

Read more

  • Kotlin vs Java
  • Producer / Consumer on Kafka and Kotlin

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *