Landmark recognition with machine learning

Sometimes you leaf through a book or a magazine, you see a beautiful place or building, but you don’t know where it is or what it’s called. An application that recognizes it from a photo would be useful here.

Using machine learning to create such an application is quite simple. This will be discussed in this article.

About Landmark Recognition

Landmark recognition service allows you to get the name of the attraction, its coordinates (longitude and latitude) and even evaluate the reliability of the original image. The confidence level will be determined when uploading an image for recognition. The greater the confidence, the more likely it is that the landmark in the original image will be recognized. All of this data can be used to personalize the app for users.

When using Landmark Recognition, the device calls the cloud API and the discovery algorithm model runs in the cloud. When using the service, the device must be connected to the Internet.

Training

Setting up the development environment

1. Create an app in AppGallery Connect

For details see Quick Guide to Android.

2.Turn on ML Kit

Click hereto learn more.

3.Upload the agconnect-services.json file, which is automatically generated after the application is created. Copy it to the app folder of your Android Studio project.

4. Set the address of the Maven repository for HMS Core SDK.

5.Integrate the Landmark Recognition SDK.

Настройте SDK в файле build.gradle в папке app.
// Import the landmark recognition SDK.
implementation 'com.huawei.hms:ml-computer-vision-cloud:2.0.5.304'

При необходимости добавьте конфигурацию плагина AppGallery Connect одним из следующих способов.

Способ 1. Добавьте следующий код в объявление в заголовке файла:
apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'

Способ 2. Добавьте конфигурацию плагина в блок плагинов.
plugins {
    id 'com.android.application'
    id 'com.huawei.agconnect'
}

Code Development

1.Get permission to use the camera.

Be sure to set a static resolution.

<uses-permission android:name="android.permission.CAMERA" />

Also be sure to get a dynamic resolution.

ActivityCompat.requestPermissions(
        this, new String[]{Manifest.permission. CAMERA
}, 1);

2.Set up an API key. The service runs in the cloud, which means that the application requires an API key to set up cloud authentication data. It is necessary to set up an API key, otherwise an error will occur while the application is running.

// Set the API key to access the on-cloud services.
private void setApiKey() {
// Parse the agconnect-services.json file to obtain its information.
AGConnectServicesConfig config = AGConnectServicesConfig.fromContext(getApplication());
// Sets the API key.
MLApplication.getInstance().setApiKey(config.getString("client/api_key"));

    }

3.Create a point of interest analyzer using one of the following methods.

// Method 1: Use default parameter settings.
MLRemoteLandmarkAnalyzer analyzer = MLAnalyzerFactory.getInstance().getRemoteLandmarkAnalyzer();
// Method 2: Use customized parameter settings through the MLRemoteLandmarkAnalyzerSetting class.

/**
 * Use custom parameter settings.
 * setLargestNumOfReturns indicates the maximum number of recognition results.
 * setPatternType indicates the analyzer mode.
 * MLRemoteLandmarkAnalyzerSetting.STEADY_PATTERN: The value 1 indicates the stable mode.
 * MLRemoteLandmarkAnalyzerSetting.NEWEST_PATTERN: The value 2 indicates the latest mode.
 */
private void initLandMarkAnalyzer() {
    settings = new MLRemoteLandmarkAnalyzerSetting.Factory()
            .setLargestNumOfReturns(1)
            .setPatternType(MLRemoteLandmarkAnalyzerSetting.STEADY_PATTERN)
            .create();
    analyzer = MLAnalyzerFactory.getInstance().getRemoteLandmarkAnalyzer(settings);
}

4.Convert an image taken with a camera or stored on a device to a bitmap. The Landmark Recognition SDK doesn’t have this feature, so you need to do it in another service.

// Select an image.
private void selectLocalImage() {
    Intent intent = new Intent(Intent.ACTION_PICK, null);
    intent.setDataAndType(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, "image/*");
    startActivityForResult(intent, REQUEST_SELECT_IMAGE);
}

Enable the Landmark Recognition service in the callback function.

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    super.onActivityResult(requestCode, resultCode, data);
    // Image selection succeeded.
    if (requestCode == REQUEST_SELECT_IMAGE && resultCode == RESULT_OK) {
        if (data != null) {
            // Obtain the image URI through getData().              imageUri = data.getData();
// Implement the BitmapUtils class by yourself. Obtain the bitmap of the image with its URI.
bitmap = BitmapUtils.loadFromPath(this, imageUri, getMaxWidthOfImage(), getMaxHeightOfImage());
        }
        // Start landmark recognition.
        startAnalyzerImg(bitmap);
    }
}

5.Start Landmark Recognition after getting the bitmap. Since the service runs in the cloud, a poor connection can slow down data transfers. Therefore, it is recommended to add a mask to the bitmap before running OCR.

// Start landmark recognition.
private void startAnalyzerImg(Bitmap bitmap) {
    if (imageUri == null) {
        return;
    }
    // Add a mask.
    progressBar.setVisibility(View.VISIBLE);
    img_analyzer_landmark.setImageBitmap(bitmap);

    // Create an MLFrame object using android.graphics.Bitmap. JPG, JPEG, PNG, and BMP images are supported. It is recommended that the image size be greater than or equal to 640 x 640 px.
    MLFrame mlFrame = new MLFrame.Creator().setBitmap(bitmap).create();
    Task<List<MLRemoteLandmark>> task = analyzer.asyncAnalyseFrame(mlFrame);
    task.addOnSuccessListener(new OnSuccessListener<List<MLRemoteLandmark>>() {
        public void onSuccess(List<MLRemoteLandmark> landmarkResults) {
            progressBar.setVisibility(View.GONE);
            // Called upon recognition success.
            Log.d("BitMapUtils", landmarkResults.get(0).getLandmark());
        }
    }).addOnFailureListener(new OnFailureListener() {
        public void onFailure(Exception e) {
            progressBar.setVisibility(View.GONE);
            // Called upon recognition failure.
            // Recognition failure.
            try {
                MLException mlException = (MLException) e;
                // Obtain the result code. You can process the result code and set a different prompt for users for each result code.
                int errorCode = mlException.getErrCode();
                // Obtain the error message. You can quickly locate the fault based on the result code.
                String errorMessage = mlException.getMessage();
                // Record the code and message of the error in the log.
                Log.d("BitMapUtils", "errorCode: " + errorCode + "; errorMessage: " + errorMessage);
            } catch (Exception error) {
                // Handle the conversion error.
            }
        }
    });
}

Application testing

The operation of the service is shown below using the example of the Oriental Pearl Tower in Shanghai.

Notes

1. Before recognizing a point of interest, set up an API key to set up cloud authentication information for the app. Otherwise, an error will occur while the application is running.

2. Landmark recognition is performed in the cloud, so processing may take some time. It is recommended to add a mask before recognition of landmarks.

Similar Posts

Leave a Reply