Bad Apple on desktop icons – working with WinAPI

If something exists, you can run Bad Apple on it
Rule 86

Over the past 15 years, Bad Apple has launched a lot of things – on homemade RISC-V processoron oscilloscopeon apples. Let’s try running Bad Apple on desktop icons using Windows API calls and a few others.

Requirements

Visual Studio 2022

Required load for Visual Studio:

winget install Microsoft.DotNet.SDK.7

winget install --id=Gyan.FFmpeg  -e

winget install yt-dlp

What will we display on?

Icon colors

We will render the video in 2 colors – black and white. There are two options for changing the file icon:

• If the file is an image, then change the image, it will be displayed in a miniature version

• Change the file extension so that Windows selects the icon we need.

The first option requires rewriting the contents of the file each time, this is a very expensive operation. Let’s use the second option.

Let’s come up with 2 file extensions, for each of which Windows will display its own icon. For example: .baclrw – white color, and .baclrb – black. Now you need to register icons for these extensions in the registry.

Let’s create 2 icons in the format .ico in folderC:\badappleresources with black and white solid filling. Let’s create and run the file badapple.reg with this content:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\.baclrb]
[HKEY_CLASSES_ROOT\.baclrb\DefaultIcon]
@="C:\\badappleresources\\badapple_black.ico"

[HKEY_CLASSES_ROOT\.baclrw]
[HKEY_CLASSES_ROOT\.baclrw\DefaultIcon]
@="C:\\badappleresources\\badapple_white.ico"

As a result, Windows will begin to correctly display files with our extensions:

Icon sizes

All desktop icons are displayed with outer indentation. It is necessary to select an icon size that will maximize the filling of the icon.

The desktop in Windows NT is practically the same Explorer window as any folder. This means that you can work with it like with any other folder – extract information about the location of icons, their parameters, etc.

In order to change the size of desktop icons using Win32, you need to make several COM calls:

• Get IFolderView2 – interface of the folder that is displayed in the desktop window.

• Call method IFolderView2::GetViewModeAndIconSize to get the current display mode and icon size

• Call method IFolderView2::SetViewModeAndIconSize to set a new icon size

CoInitialize(NULL); // Запускаем COM
CComPtr<IFolderView2> spView;
FindDesktopFolderView(IID_PPV_ARGS(&spView)); //Получим IFolderView2 окна рабочего стола
const auto desiredSize = 67;
FOLDERVIEWMODE viewMode;
int iconSize;
spView->GetViewModeAndIconSize(&viewMode, &iconSize);
spView->SetViewModeAndIconSize(viewMode, desiredSize);

void FindDesktopFolderView(REFIID riid, void** ppv)
{
    CComPtr<IShellWindows> spShellWindows;
    // Создаем экземпляр IShellWindows
    spShellWindows.CoCreateInstance(CLSID_ShellWindows); 

    CComVariant vtLoc(CSIDL_DESKTOP);
    CComVariant vtEmpty;
    long lhwnd;
    CComPtr<IDispatch> spdisp;
    // Находим окно по его идентификатору SW (SW_DESKTOP в случае рабочего стола)
    spShellWindows->FindWindowSW(
        &vtLoc, &vtEmpty,
        SWC_DESKTOP, &lhwnd, SWFO_NEEDDISPATCH, &spdisp); 

    CComPtr<IShellBrowser> spBrowser;
    CComQIPtr<IServiceProvider>(spdisp)->
        QueryService(SID_STopLevelBrowser,
            IID_PPV_ARGS(&spBrowser));

    CComPtr<IShellView> spView;
    // Находим активный IShellView в выбранном окне
    spBrowser->QueryActiveShellView(&spView); 
    // Находим выбранный объект (в нашем случае IFolderView2) в IShellView
    spView->QueryInterface(riid, ppv);
}

Adjusting the value desiredSize We are looking for an icon size such that the margins along the edges of the icon will be minimal. This value is 67.

This code will resize the icons to 67 each time the program is launched.

“Virtual screen” resolution

Now you need to find out how many icons fit on the desktop horizontally and vertically. You can calculate it manually (an uninteresting option), or you can calculate it automatically after adjusting the size of the icons (a more interesting option):

RECT desktop;
// Получаем HANDLE окна рабочего стола
HWND hDesktop = GetDesktopWindow();
// Получаем прямоугольник окна рабочего стола
GetWindowRect(hDesktop, &desktop);

POINT spacing;
//Получаем ширину значка вместе с отступами
spView->GetSpacing(&spacing);
auto xCount = desktop.right / spacing.x;
auto yCount = desktop.bottom / spacing.y;

IFolderView2::GetSpacing returns the dimensions of the icon taking into account indentations (which is what we need). We divide the screen side by the icon side and get the number of icons per side, on my 2560×1440 pixel monitor that’s 34×11 icons. Remember these numbers, they will come in handy later.

What will we display?

Video preparation

Let’s prepare the necessary files. Download the original video:

yt-dlp "https://www.youtube.com/watch?v=FtutLA63Cp8" -o "badapple.mp4"

It is necessary to reduce the video resolution to 34×11, and the frequency to 10 frames per second:

ffmpeg -i badapple.mp4 -s 34x12 -c:a copy -filter:v fps=10 downscaled-33x12.mp4

Pay attention to the specified resolution – 34 by 12. Unfortunately, ffmpeg requires an even number of vertical pixels, you will have to ignore the last line.

Converting Video to Pixels

You can read the entire video frame by frame and save the pixel value into a binary file. However, at this point you can make a small optimization in terms of code but significant in terms of performance.

Let’s make the application render only the changed pixels. To do this, we will store the previous frame in a buffer and write only the data of the changed pixels to a binary file. To do this, we will write an application in C#; to read the video we will use the package GleamTech.VideoUltimate.

using System.Runtime.CompilerServices;
using GleamTech.Drawing;
using GleamTech.VideoUltimate;

// Ширина входного видео
const int WIDTH = 34;
// Высота входного видео (с учетом игнорируемой последней строки)
const int HEIGHT = 11;

// Значение байта для БЕЛОГО пикселя
const byte BYTE_ONE = 255;
// Значение байта для ЧЕРНОГО пикселя
const byte BYTE_ZERO = 0;
// Значение байта для команды "ПОКРАСИТЬ ПИКСЕЛЬ"
const byte BYTE_FRAME_PIXEL = 0;
// Значение байта для команды "Сделать скриншот"
const byte BYTE_FRAME_SCREENSHOT = 1;

// Ссылка на выходной файл
var outputPath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "framedata.bapl");
// Поток выходного файла
using var outputStream = File.Open(outputPath, FileMode.Create, FileAccess.ReadWrite);
// Входной файл
using var videoReader = new VideoFrameReader(File.OpenRead("downscaled-33x12.mp4"));

// Буфер предыдущего кадра
var buffer = new Span<bool>(new bool[WIDTH * HEIGHT]);

//Считываем кадры, пока они доступны
for (var i = 0; videoReader.Read(); i++)
{
    Console.WriteLine(i);
    //Получаем кадр и преобразуем его к Bitmap из .NET
    using var frame = videoReader.GetFrame();
    using var bitmap = frame.ToSystemDrawingBitmap();
    for (byte x = 0; x < WIDTH; x++)
    {
        for (byte y = 0; y < HEIGHT; y++)
        {
            //Индекс пикселя в буфере
            var bufferValue = WIDTH * y + x;
            //Получим значение пикселя (канал значения не имеет, видео черно-белое)
            var color = bitmap.GetPixel(x, y).R > 128;
            if (buffer[bufferValue] != color)
            {
                // Записываем байт команды изменения пикселя
                outputStream.WriteByte(BYTE_FRAME_PIXEL);
                // Записываем данные измененного пикселя
                outputStream.WriteByte(x);
                outputStream.WriteByte(y);
                outputStream.WriteByte(color ? BYTE_ONE : BYTE_ZERO);
                buffer[bufferValue] = color;
            }
        }
    }
    //Записываем байт команды скриншота
    outputStream.WriteByte(BYTE_FRAME_SCREENSHOT);
}

The converter watches the video frame by frame and compares the pixels with the buffer of the previous frame. If the color value is different, then the pixel drawing command is written to the output file:

• Byte BYTE_FRAME_PIXEL(0)

• Coordinate X pixel

• Coordinate Y pixel

• MeaningBYTE_ONE or BYTE_ZERO – white or black pixel color.

Before completing frame processing, a byte is written to the output file BYTE_FRAME_SCREENSHOT(1) – the renderer will take a screenshot of the desktop when it encounters this byte.

As a result of the converter’s operation, a file appears on the desktop framedata.baplwhich contains the order of drawing pixels and taking a screenshot of the desktop.

How will we display

Let’s fill the desktop with files with black icons. To do this, get the path to the current user’s desktop using SHGetSpecialFolderPathA:

// desktopPath будет указывать на строку wstring - путь к рабочему столу текущего пользователя
  char desktop_path[MAX_PATH + 1];
  SHGetSpecialFolderPathA(HWND_DESKTOP, desktop_path, CSIDL_DESKTOP, FALSE);

  const auto totalScreenCapacity = desktopResolution.x * desktopResolution.y;
  auto desktopPath = std::string(desktop_path);
  auto desktopPathW = std::wstring(desktopPath.begin(), desktopPath.end());

Next, let’s fill in 2 vectors – the full paths to the files of white flowers and black flowers.

for (auto y = 0; y < desktopResolution.y; y++)
    {
        for (auto x = 0; x < desktopResolution.x; x++)
        {
            blacks[y * desktopResolution.x + x] = desktopPathW + line_separator + std::to_wstring(x) + L"_" + std::to_wstring(y) + black_extension;
            whites[y * desktopResolution.x + x] = desktopPathW + line_separator + std::to_wstring(x) + L"_" + std::to_wstring(y) + white_extension;
        }
    }

As a result, in blacks And whites there will be lines like C:\Users\[User]\Desktop\[x]_[y].baclr['w'|'b'] .

We will change the file icon by renaming it.

Renaming a file

To rename a file, the Win32 API provides a function MoveFile. We could change the index icon i from white to black like this:

MoveFile(whites[i], blacks[i]);

However, this approach has a significant drawback – with each such rename, a new file descriptor will be created and deleted, and creating a descriptor is a very expensive operation.

We can use a slightly different approach – create a file using the function CreateFilesave the resulting descriptor into a vector and use it for future renaming without closing the file.

 for (int i = 0; i < totalScreenCapacity; i++) {
    // Создаем файлы с черной иконкой с доступом на чтение, запись и удаление
    handles[i] = CreateFile(blacks[i].c_str(), GENERIC_READ | GENERIC_WRITE | DELETE, 0, NULL, CREATE_ALWAYS, 0, NULL);
}

All that remains is to directly rename the file, knowing it HANDLEfor this there is SetFileInformationByHandle. Let’s write a function:

void RenameFileByHandle(HANDLE handle, std::wstring newName) {
    auto newNameStr = newName.c_str();
    // Создадим структуру с информацией о длине файла
    union
    {
        FILE_RENAME_INFO file_rename_info;
        BYTE buffer[offsetof(FILE_RENAME_INFO, FileName[MAX_PATH])];
    };

    file_rename_info.ReplaceIfExists = TRUE;
    file_rename_info.RootDirectory = nullptr;
    //Заполним информацию о длине названия файла
    file_rename_info.FileNameLength = (ULONG)wcslen(newNameStr) * sizeof(WCHAR);
    // Запишем нули в название файла (для нормальной работы SetFileInformationByHandle название файла должно кончаться на \0)
    memset(file_rename_info.FileName, 0, MAX_PATH);
    // Скопируем нужное название файла в память
    memcpy_s(file_rename_info.FileName, MAX_PATH * sizeof(WCHAR), newNameStr, file_rename_info.FileNameLength);

    // Переименуем файл
    SetFileInformationByHandle(handle, FileRenameInfo, &buffer, sizeof buffer);
}

Taking screenshots

We will use GDI+let’s write a function SaveScreenshotToFile.

void SaveScreenshotToFile(const std::wstring& filename)
{
    // Получим контекст устройства экрана
    HDC hScreenDC = CreateDC(L"DISPLAY", NULL, NULL, NULL);

    // Получим размер экрана
    int ScreenWidth = GetDeviceCaps(hScreenDC, HORZRES);
    int ScreenHeight = GetDeviceCaps(hScreenDC, VERTRES);

    // Создадим изображение
    HDC hMemoryDC = CreateCompatibleDC(hScreenDC);
    HBITMAP hBitmap = CreateCompatibleBitmap(hScreenDC, ScreenWidth, ScreenHeight);
    HBITMAP hOldBitmap = (HBITMAP)SelectObject(hMemoryDC, hBitmap);

    // Скопируем скриншот из контекста экрана в контекст памяти (изображения)
    BitBlt(hMemoryDC, 0, 0, ScreenWidth, ScreenHeight, hScreenDC, 0, 0, SRCCOPY);
    hBitmap = (HBITMAP)SelectObject(hMemoryDC, hOldBitmap);

    // Сохраним изображение в файл
    BITMAPFILEHEADER bmfHeader;
    BITMAPINFOHEADER bi;

    bi.biSize = sizeof(BITMAPINFOHEADER);
    bi.biWidth = ScreenWidth;
    bi.biHeight = ScreenHeight;
    bi.biPlanes = 1;
    bi.biBitCount = 32;
    bi.biCompression = BI_RGB;
    bi.biSizeImage = 0;
    bi.biXPelsPerMeter = 0;
    bi.biYPelsPerMeter = 0;
    bi.biClrUsed = 0;
    bi.biClrImportant = 0;

    DWORD dwBmpSize = ((ScreenWidth * bi.biBitCount + 31) / 32) * 4 * ScreenHeight;

    HANDLE hDIB = GlobalAlloc(GHND, dwBmpSize);
    char* lpbitmap = (char*)GlobalLock(hDIB);

    // Скопируем биты изображения в буффер
    GetDIBits(hMemoryDC, hBitmap, 0, (UINT)ScreenHeight, lpbitmap, (BITMAPINFO*)&bi, DIB_RGB_COLORS);

    // Создадим файл с будущим скриншотом
    HANDLE hFile = CreateFile(filename.c_str(), GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);

    // Размер в байтах заголовка изображения
    DWORD dwSizeofDIB = dwBmpSize + sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);

    // Сдвиг данных пикселей
    bmfHeader.bfOffBits = (DWORD)sizeof(BITMAPFILEHEADER) + (DWORD)sizeof(BITMAPINFOHEADER);

    //Размер файла
    bmfHeader.bfSize = dwSizeofDIB;

    //0x4d42 = 'BM' в кодировке ASCII, обязательное значение
    bmfHeader.bfType = 0x4D42; //BM   

    DWORD dwBytesWritten = 0;
    WriteFile(hFile, (LPSTR)&bmfHeader, sizeof(BITMAPFILEHEADER), &dwBytesWritten, NULL);
    WriteFile(hFile, (LPSTR)&bi, sizeof(BITMAPINFOHEADER), &dwBytesWritten, NULL);
    WriteFile(hFile, (LPSTR)lpbitmap, dwBmpSize, &dwBytesWritten, NULL);

    //Очищаем данные контекстов
    GlobalUnlock(hDIB);
    GlobalFree(hDIB);

    //Закрываем файлы
    CloseHandle(hFile);

    //Очищаем мусор после себя
    DeleteObject(hBitmap);
    DeleteDC(hMemoryDC);
    DeleteDC(hScreenDC);
}

Before taking a screenshot, you need to refresh the contents of the window and wait until the update occurs. For this we will use SendMessage and write a small function:

void TakeScreenshot(int index){
    // Путь i-того скриншота
    auto path = screenshot_path + L"shot_" + std::to_wstring(index) + L".png";
    // Отправляем сообщение для обновления
    SendMessage(HWND_BROADCAST, WM_SETTINGCHANGE, 0, 0);
    // Ждем DEFAULT_SLEEP_TIME миллисекунд
    std::this_thread::sleep_for(DEFAULT_SLEEP_TIME);
    // Делаем скриншот
    SaveScreenshotToFile(path);
}

Team SendMessage is intended to send a message to the selected window (0 argument), we use HWND_BROADCAST, so all windows will be the recipients. In our case the message is WM_SETTINGCHANGE – change window settings. This message causes windows to re-render themselves and their child windows.

Almost final – putting it all together

Function main The renderer will look something like this:

int main()
{
    // Получаем параметры рабочего стола
    auto desktopResolution = GetDesktopParams();
    const auto totalScreenCapacity = desktopResolution.x * desktopResolution.y;

    // Создаем файлы и заполняем векторы с названиями файлов и дескрипторами
    std::vector<HANDLE> handles(totalScreenCapacity);
    std::vector<std::wstring> blacks(totalScreenCapacity);
    std::vector<std::wstring> whites(totalScreenCapacity);
    FillDesktop(desktopResolution, handles, blacks, whites);

    // Считываем содержимое файла framedata.bapl
    auto bytes = ReadAllBytes(pixel_source_path);
    auto i = 0; 
    auto frame = 0;

    // Отрисовываем созданные файлы
    SendMessage(HWND_BROADCAST, WM_SETTINGCHANGE, 0, 0);
    
    while (i < bytes.size()) {
        i++;
        //Считываем очередной байт
        char value = bytes[i];
        // Если это команда для снимка экрана - делаем скриншот
        if (value == BYTE_FRAME_SCREENSHOT) {
            TakeScreenshot(frame);
            frame++;
        }
        else {
            // Получаем координаты и цвет пикселя
            auto x = bytes[i + 1];
            auto y = bytes[i + 2];
            auto color = bytes[i + 3];
            i += 3;

            // Переименовываем соответствующий файл
            auto position = y * desktopResolution.x + x;
            RenameFileByHandle(handles[position], color == BYTE_ONE ? whites[position] : blacks[position]);
        }
    }
    // Делаем финальный скриншот
    TakeScreenshot(frame);
    return 0;
}

The renderer reads the file framedata.bapl byte by byte, renames the corresponding file or takes a screenshot at the right moment. At the output we get many files of the format .bmp – screenshots of the window for each frame of the Bad Apple video.

Assembling screenshots back into videos

There’s still a little time left, we’ll get to the video soon)

We use ffmpeg:

ffmpeg -framerate 10 -i "scan_%d.bmp" output.mp4

All that remains is to add sound in your favorite video editor.

The final

Results

As part of the article, we were able to run Bad Apple on desktop icons. During the work, we used many Windows API functions, learned how to access some COM interfaces, take screenshots using GDI+ and compose them into videos using ffmpeg.

Link to project repository