When private, but really want public

In 2016, I was invited to help develop “ORBI” action glasses; these are waterproof glasses with several cameras that can stream 360 video directly to a smartphone, and if you swim with them, nothing should break either. (https://www.indiegogo.com/projects/orbi-prime-the-first-360-video-recording-eyewear#/). Actually, my task was to write an algorithm for gluing a video stream from four cameras into one large 360* video, at that time the task was not very difficult, but required a little specific knowledge of opencv and the environment. But the article is not about that, because now it’s all protected IP, but about how we wrote a test environment for the classes used and, accordingly, the algorithms using legal and not so legal means of the C++ language. Yes, you say, what’s wrong with it – he made heterosexuals and test it for your health. And if there is no getter or the class variable is hidden in a private section and there is no possibility to change the header. Or did the lib vendor forget to include headers and only sent a scan of the source code (Chinese friends are like that), but this needs to be tested? By multiplying the desire to write tests for morning coffee and adding wild enthusiasm, you can get a lot compilation errors interesting experience. If you can’t, but really want to, then you can. As one leader I know said: “There is no code that we cannot refactor, especially over morning coffee.”


At the initial stage of development, we didn’t think much about the overall architecture; we assembled a prototype from free software and some mother opencv + hugin + stitchEm, that’s why the tests were scattered throughout the project, with inclusions of varying degrees of harmfulness. Well, the main thing is that at least they were there and launched.
By the time the prototype was presented at the end of 2016, one of the main classes of the device, which assembled 4 frames into one, was hung with tests, I can’t remember exactly, but something like 50 friends for tests. I repeat, this was not done on purpose, it happened historically, and then became common practice. Each week of development added a couple more test friends to this class, and at some point they decided that this bad practice needed to be removed.

I'll show it to you now.  Here he is, this insidious type of civilian appearance!

I’ll show it to you now. Here he is, this insidious type of civilian appearance!

I’ll show it to you now. Here he is, this insidious type of civilian appearance!
class OrbiVideo421 : public orbi::intrusive_ptr {
  
	friend struct TestVideoStream;
	friend struct TestVideoFrame; 
	friend struct TestVideoStitcher; 
	friend struct TestVideoPlayer; 
	friend struct TestVideoTest; 
	friend struct TestVideoStreamProcessor; 
	friend struct TestVideoCapture; 
	friend struct TestVideoWriter; 
	friend struct TestVideoProcessor; 
	friend struct TestVideoTestSuite;
	friend struct TestVideoAnalyzer; 
	friend struct TestVideoConfiguration; 
	friend struct TestVideoBuffer; 
	friend struct TestVideoRenderer; 
	friend struct TestAudioStream; 
	friend struct TestAudioFrame; 
	friend struct TestAudioStitcher; 
	friend struct TestAudioPlayer; 
	friend struct TestPerformanceAnalyzer; 
	friend struct TestErrorLogger; 
	friend struct TestVideoFrameAnalyzer; 
	friend struct TestVideoSynchronization; 
	friend struct TestVideoMetadataExtractor; 
	friend struct TestVideoComparison; 
	friend struct TestVideoStreamController; 
	friend struct TestVideoStreamEncoder; 
	friend struct TestVideoStreamDecoder; 
	friend struct TestVideoStreamRouter;
	friend struct TestVideoEffect;
	friend struct TestVideoSegmentation; 
	friend struct TestVideoStreamRecorder; 
	friend struct TestVideoCompression;
	friend struct TestVideoStreamSplitter; 
	friend struct TestVideoStreamSwitcher; 
	friend struct TestVideoStreamMetadataEditor; 
	friend struct TestVideoStreamAnalyzer; 
	// ниже еще около 40 френдов для тестов
}

That’s right, long and expensive

class	OrbiMemoryUnit : orbi::vtable_at_top {
  friend MemoryTestRead;
  friend MemoryTestWrite;
  
  private:
    nobject *_object;
    uint32_t _reserved_memory;
}

Everything was hidden in internal structures; individual test friends used them at their own discretion. But then we, the beautiful ones, appeared and decided to do it according to science, adding hetaeras to the required members of the class. Of course, people helped not only to add getters, but also to slightly refactor the code and names. A couple of weeks of monkey work loomed on the horizon, and it’s not a fact that it will be useful, because the class interface is starting to change for the sake of tests. In addition, the “breaking changes” rule is violated – changing an established API without a good reason means getting yourself problems in the future with compatibility and side effects that you won’t catch. When a refactor arrives at the review in half with changed variable names for 300+ lines, it was very painful to watch. And in the end, after a couple of calls, they stopped this practice, rolled back the changes and sat down to think about how to do it differently.

class	OrbiMemoryUnit : orbi::vtable_at_top {
  private:
    nobject *_object;
    uint32_t _reserved_memory;

  public:
  	inline	nobject					*object					() const	{ return _object; }
	inline	uint32_t				reserved_capacity 		() const	{ return _reserved_memory; }
}

The advantages of this approach are obvious, everything is within the framework of the language model, understandable to newcomers to the project. The disadvantages are not so obvious, or maybe they are not even disadvantages, the most obvious is the need to think through a normal API, which on small projects with limited resources and time stretches out the rollout of features and takes time away from the team – which was stated to me by the PM at one of the meetings . In general, it’s correct – but it’s time-consuming and expensive, so it’s wrong.

Black magic of macros, but sometimes it doesn’t compile

 Some kind of strange tuning table you have - in circles!

Some kind of strange tuning table you have – in circles!

So we lived with this heritage, until one fine morning our colleague from sunny Catania came across one of the classes, who had recently joined this project module, deciding to add something there. He was a very unittest friendly pogromist, so the number of tests increased one and a half times in the morning. And already at lunchtime a congratulation message arrived in the mail of the build engineer on the fact that the number of friends in the class OrbiDeviceBattery exceeded 100 pieces (we later found this out experimentally) and the Keil-C compiler failed to compile it, dumping this error in the log.

C:\ent\orbi\prod\KeilMDK\INCLUDE\keillex\xstring(25) : error C2146: syntax error : missing ';' before identifier 'friend'
C:\ent\orbi\prod\KeilMDK\INCLUDE\keillex\xstring(597) : see reference to class template instantiation 'OrbiDeviceBattery<_E,_Tr,_A>' being compiled
C:\ent\orbi\prod\core\battery\device.cc(655) : error C2838: illegal qualified name in member declaration

How about a non-template class? OrbiDeviceBattery became a template, and why it affected xstring is unclear. The words in the error log have little to do with the error itself, we just broke the compiler. I wrote to the support of the compiler developer and did not receive a clear answer, but after wandering around the forum I saw that similar complaints to the core compiler team have an average closure time of six months or more, if you go to the Keil forum you can find a couple that were opened back in 2017 and before The status is still “unfixed”. Apparently it’s been there for five years, and will be there for another five. On line 655, as expected, there was another class friend.

It won't be enough!

It won’t be enough!

After creaking for a while with gray cells that cannot be restored, we found a completely working hack for a test environment. You can override private/class to public/struct. It seems that this is “happiness” – write tests to your heart’s content, without any friends. But even here the pink bird Oblomingo was waiting for us, what is assembled on clang will not necessarily work on another clang. The Keil compiler happily reported that we are trying to redefine keywords, which, as it were, should not be done at all.

#define private public // illegal
#define class struct // illegal

#include "core/view_direction.h"
#include "core/view_vec2i.h"
#include "core/view_point.h"

But gcc/clang completely allows this (https://onlinegdb.com/pag5bTK2Yv). But you shouldn’t do this at all, it’s the same as raising Public Morozovs in your project. It was we who suffered from hopelessness. Make normal decisions within the framework of the language model, it will come back with less technical debt, believe my experience.

#include <iostream>

using namespace std;

#define private public
#define class struct

class A { int l; };

int main() {
    A a;
    cout << a.l;
    return 0;
}

Grandfather of Public Morozov

But the problem no longer wanted to let go of the restless brain, especially since the startup received the next round of funding and it was possible to relax a little and straighten your back, do something more useful, besides writing these hated tests and coming up with gluing algorithms. Herb Suttter wrote about the problem of access rights violations when working with templates back in 2010 (http://www.gotw.ca/gotw/076.htm), but considered it more of a feature of the language for every fireman.

He even had a separate rule for this case:

never subvert the language; for example, never attempt to break encapsulation by copying a class definition and adding a friend declaration, or by providing a local instantiation of a template member function (GotW #76), Herb Sutter

Here is an example from Sutter himself (https://onlinegdb.com/cn7bOdWdn), how you can break the encapsulation mechanism of pluses, with one caveat, the class must have a free template function in the public zone. Everything works quite simply, instantiating a class template function gives us access to the private data of the class, because in fact it is a class function with all rights.

class X { 
public:
  X() : private_(1) { /*...*/ }

  template<class T>
  void f( const T& t ) { /*...*/ }

  int Value() { return private_; }

private: 
  int private_; 
};

namespace {
  struct Y {};
}

template<>
void X::f( const Y& ) {
  private_ = 2; // evil laughter here
}

int main() {
  X x;
  cout << x.Value() << endl; // prints 1
  x.f( Y() );
  cout << x.Value() << endl; // prints 2
}

In principle, we could have stopped there, because in 90% of cases this covers the testing needs, and our friends become simply free classes that will be a parameter for such a proxy function. Minimum functionality changes, maximum benefits, i.e. minus all test friends from the header. If not for one BUT… not everywhere we could add such a template function to pull off this trick.

The gardener killed everyone

After experimenting a little more with obtaining private class members, it became clear that none of the existing compilers is able to obtain the address of a class member if it is cast to another data type that is not private. If we try to get the address of a private variable, we will get a compiler error, which is logical, the compiler is smarter than us and protects us from shots in different parts of the body.

struct A {
private:
  char const* x = "private data";
};

int main() {
  auto ptr = &A::x;
}

...

main.cpp: In function ‘int main()’:
main.cpp:46:19: error: ‘const char* A::x’ is private within this context
   46 |    auto ptr = &A::x;
      |                   ^
I brought you a parcel.  I won’t give it to you, because you don’t have documents.

I brought you a parcel. I won’t give it to you, because you have documents No.

But if we carefully ask the compiler to give the ability to shoot wherever you want the address of this variable as a template parameter, then everything will be fine. We don’t do anything further with it, we just indulge in buns.

struct A {
private:
  char const* x = "private data";
};

template <class Stub, typename Stub::type x>
struct private_member_x {
    private_member_x() { auto ptr = x; }
};

// так мы просим компилятор не обращать внимания на реальный тип переменной
struct A_x { typedef char const *(A::*type); };

// и теперь компилятор считает, что работает с другим типом (подменным)
template struct private_member_x<A_x, &A::x>;

int main()
{}

// все чисто, никаких ошибок компиляции

So, we have the address of the private variable; there is very little left to write a wrapper for storing and working with these addresses. For example something like this (compilable) https://onlinegdb.com/MERIajJcH

// шаблон для класса заглушки, который будет хранить адрес переменной
template <class Stub>
struct member { static typename Stub::type value; }; 

// статик переменная для общего шаблона
template <class Stub> 
typename Stub::type member<Stub>::value;

// подмена типа и получение адреса приватной переменной
// stub уйдет дальше в класс member, чтобы имять статик переменную для этого типа
template <class Stub, typename Stub::type x>
struct private_member {
  private_member() { member<Stub>::value = x; } // сохранение адреса переменной
};

// тесткейс
struct PapaPavlica {
private:
  char const* papini_dengi = "papini dengi";
};

// подменный тип, чтобы компилятор не пугался нарушением прав доступа
struct A_x { typedef char const *(PapaPavlica::*type); };

// магия, здесь живут драконы
template struct private_member<A_x, &PapaPavlica::papini_dengi>;

int main() {
   PapaPavlica papa;
   // разворачиваем обратно полученный адрес в переменную
   // получаем *(papa).(&PapaPavlica::papini_dengi) = "deneg net"
   papa.*member<A_x>::value = "deneg net";
   std::cout << papa.*member<A_x>::value << std::endl; // deneg net
}

We did not put this code into production; it remained the lot of the test environment, but it fit perfectly there. And of course, I won’t leave you without a working library (https://github.com/altamic/privablic), buy beer for our unit test lover from sunny Catania when you meet him.

If you can’t, but really want to, then you can

If you can’t, but really want to, then you can

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *