I see you littering from your balcony

So you go out to work in the morning, and on the lawn in front of the house there are cigarette butts and other waste products. It happens that you come home, and you have ashes or even a cigarette butt on your balcony. How would you like to know who did it? I can’t stand such people. My story is about how to find pests and maybe make your home a little cleaner and safer.

My house is a classic human settlement, without underestimation, with a thousand devils man inside. Don’t think I love my home. To some extent, I even know a couple of dozen neighbors. Good people. But it happens that you come home and see such a picture on the balcony.

Or maybe you come back from vacation, and you have this.

I’ll start with a little background. At the beginning of summer, the heat continued for several weeks. A friend called me and asked me to go check his apartment, he was told that there was a fire in the house. They were not in the city at that time with their family.

I arrived quickly as I live nearby. Smoke billowed from every window on their floor. I walked around the house to see the picture from the side of the balconies. Unfortunately, my friends’ apartment was on fire. Apart from their balcony, the one next door was also on fire.

The worst thing is that there was a cat in the apartment that I looked after. I went downstairs back and forth and could not do anything.

Lots of fire trucks. The house is located so that firefighters can not directly extinguish the fire. The road is blocked or something else – there was no time to figure it out. Firefighters from neighboring windows tried to flood the apartment with water. At times like this, everything seems to go very slowly. I’m not a firefighting expert, but it looked very inefficient, but who am I to judge them, I don’t put out fires.

Firefighters must check that no one is in the room. There was no one. Visibility is zero.

As a result, when the fire was extinguished and the smoke cleared, I was allowed to get up. Basically, the balcony and the adjoining room burned down. A false ceiling hung from the ceiling throughout the apartment like stalactites in a cave. Every firefighter said that it was impossible to survive in such a fire. By some miracle, the cat managed, she huddled under the sofa, spent hours in complete hell. Fear mixed with anger towards everything around, scorched foggy white eyes… The cat didn’t deserve all this.

There are two official versions of the fire. Careless handling of fire when smoking and a version associated with another source of fire on the street. In both versions, something burning is blown onto the balcony by gusts of wind.

Could it have been avoided? Certainly! Close windows and that’s it. On the other hand, when leaving for work, will you close all windows when your beloved pet is at home? I doubt. Mosquito nets could save, but, frankly, it is not always possible to install them.

At that moment, I thought, is it possible to somehow understand the reason for the fire? Is it possible to somehow prove that it was not you who forgot to turn off the iron, for example?

How to solve a problem?

Let’s go back to my house. Based on the fact that cigarette butts and other garbage appear on the lawn regularly, I would like to understand who and how often throws garbage from the front of the house. I was especially interested in smokers. After that incident with the fire, my walks with the children turned into something similar.

After several hours of observing the facade, I only realized that my neck hurt, and in order to identify those who litter, this observation needs to be automated somehow.

Approximately so I imagine the scheme of service of investigation of “facade” incidents. There must be some kind of camera that looks at the front of the house. Video from this camera gets into the service that breaks the video into zones – balconies. After that, the magic of video analysis happens, and as a result, we should get reports, and ideally, from the reports, understand who is the daddy pig in the end.

I’m watching you

Let’s start with the camera.

I would like to install a full-fledged IP camera and analyze video around the clock, but it is impossible to do this quickly. We need approvals from the tenants and the Criminal Code and, probably, a bunch of bureaucratic things that I don’t like. I have an action camera.

I put a 265 GB SD card (to fit more videos) and started recording the facade of the house in 4K. But there were a number of problems with this solution.

1. My camera does not support 256GB cards. Sometimes I am not very smart and understanding comes only after I bought a 256 GB card. Life Lesson #256: You should always read instructions. I had to use 128 or less, which I already had.

2. Even when you fix the camera on the holder and put it in the same place, the video is still recorded with an offset. It is necessary to either reshoot or change the settings of the balcony zones.

3. The camera can be stolen. She has to be followed.

For a couple of days I chose the position of the camera and made test recordings. After that, for a week every evening I went out for a walk with the children and set up a camera on which I recorded a 4K video of the facade of the house. Actually more than a week due to weather conditions and other things.

The camera itself splits the video into files, so they must first be “glued” into one large video file. After that, split the video into zones.

This is a rather rough division, and besides, it is shown schematically. The video needs to be divided into fragments, that is, each zone is a separate video. This will make it easier to analyze. Otherwise, you need to set a high sensitivity for analytics, since the sizes of moving objects can be very small. This in turn will give a large number of false positives.

You can split the video into small pieces using ffmpeg, VLC, or something else.

You take ffmpeg and do it 30 times, as shown above, and here you have a bunch of videos with resolutions from 360×240 to 704×280.

Analytics

Now each zone can be analyzed separately and generate a report. I see several analysis options:

1. Detection of objects (people)

2. Smoke and fire detection

3. Motion detection

You can do this analytics yourself, for example, based on YOLO or OpenCV (https://habr.com/ru/companies/avi/articles/200804/) or use a ready-made service. I came across a ready-made server with video analytics from Macroscop, when you work for this company, you often come across ready-made servers with video analytics.

I checked some parts of the facade videos, which clearly showed people and the fact of smoking. Unfortunately, the conditions turned out to be unsuitable for detecting objects and detecting smoke and fire, people in the video are often “cut off”, and there is too little smoke and fire to be recognized. I had to abandon this analysis.

The motion detector worked perfectly. After checking several videos, it was possible to collect a large amount of information about the intervals of movement.

Some time was spent in advance to “prepare” each zone. Various rags, banners and other randomly moving things create a large number of false positives. Zones can simply be trimmed so that there are fewer fakes.

As a result, when the camera, analytics and everything else is selected, something like this turned out.

I know what you were doing on your balcony!

How can you still identify violators by motion detection? Smokers who throw away cigarette butts – no way, but for starters, you can determine the very fact of smoking. According to my observations, cigarettes are smoked for an average of 3 to 6 minutes. So my goal will be to find all intervals with movement in that range in each zone. Ideally, find some patterns.

For 5 days, from about 18:00 to 22:00, I recorded a video of the facade and analyzed it for movement. First of all, it was interesting to determine whether there is a smoker there or not from the traffic schedules. Game – “Find a smoker.”

This is the first graph of one of the balconies that I began to closely analyze. The graph shows the intervals of movement at the initial moments during all days of observation. Roughly speaking, on Tuesday, somewhere around 18-00, someone or something moved for 2 minutes. There were some minor nuances when plotting graphs, for example, a graph can contain several zones, but this is not particularly important. Do you see any patterns? No matter how long I looked at it, I found nothing. The only thing I came to is that after 21-00 no one went out to the balcony.

As a result, I watched the video fragments corresponding to the peak values. It turned out that this is a playing child running back and forth and his mother hanging clothes on the dryer.

Balcony smokers!

I continued to analyze and review peak values. To be honest, there were not so many of them, especially those behind which smokers were hiding. I’ll tell you about some of them.

Smoker #1

Most of all, one graph caught my attention.

He went out to smoke exactly at 20-30 +- 10 minutes, once every day. It seemed to me very unusual. It seems that the probability of meeting such a chart in the morning is much higher.

Smoker #2

During the observation period, only he obviously threw cigarette butts out of the window. It is not very legal to place a person without his consent on private property, therefore, with an accuracy of 98%, he replaced a real photo with one generated using modern technologies.

Smoker photo #2

We will return to it a little later.

Smoker #3

He goes out onto the common balcony, stands, smokes, and in the same place, somewhere in the corner, throws out cigarette butts, probably in a coffee can. Nothing out of the ordinary. Looking at him, you can say that an ordinary uncle went out for a smoke and, like a decent person, threw a cigarette butt into a “special” trash can. Is it really all right?

1. There is a law prohibiting smoking on a public balcony, but my goal is not to organize a “witch hunt”, so this item can not even be considered. Actually, now it’s normal, well, let’s say how to throw a candy wrapper or chewing gum past the trash can.

2. He throws the ashes down. All the while he stands and smokes. He gets to the people on the balconies. I wonder if the ashes can start a fire?

3. The jar into which he throws cigarette butts is not recorded, here is a photo of one of the messages in the general house chat.

public balconies.

We will consider them separately. I wrote above that they go out to smoke, but children also throw off toys, water balls and cans of cigarette butts from them. For these zones, I looked at all the movements that lasted more than 1 minute, since it usually takes seconds to overcome the balcony zone. For a day, about three times, the following happens.

They throw something down, and then they do everything in the corner.

Then comes the social part of the experiment.

This city needs a new hero!

From the balcony you can easily understand in which apartment the smoker lives. Let’s meet my neighbors.

Smoker #1

It is located quite high and it is completely unclear from the video whether something is thrown out of the window or not. I have no evidence, but I really want to see his reaction. I picked up a random cigarette butt on the street and went to meet my neighbor at about 20-50, making sure that he finished smoking.

– I’m your downstairs neighbor, this cigarette butt flew to my balcony, and I also find ashes from time to time.

– It’s not me. Closes the door.

In fact, he muttered something quickly, but that was the gist of it. The reaction is expected. Maybe my visit will be delayed in his mind.

From time to time I glanced at his balcony. I’m interested. At 20-30 I didn’t see him anymore, maybe he’s on vacation or he changed the time / place of smoking. Imagine helping change his life for the better. You do not think, I am not against smokers as such. If there’s one thing I hate, it’s when they litter. Hate.

Smoker #2

Every time he smoked, he threw all the cigarette butts out into the street. I went up to the floor. I call the same apartment. A normal adequate, at first glance, little man came out. He began to talk about some workers in his apartment, and ended up talking about his children, boobies. It was some incoherent chatter. I wedged into a monologue, asked not to litter anymore, barely said goodbye and left.

Smoker No. 3 and the incident of common house balconies

I didn’t get to know anyone here. Although I had different ideas for experiments. For example, what happens if you pick up a can of cigarette butts every time? Will he throw cigarette butts off the balcony or will he take the ashtray home?

Maybe in this case, write to the general chat? As soon as you read such a thing in a chat, you immediately think to yourself: what a suffocation – at least I almost always do it. He just smokes and doesn’t even litter, every second would think! Everything needs to be proven and explained. In general, I have not decided what to do with it.

Children are bored, they climb onto the balconies. Do you know what the view is from the 25th floor? Awesome, honestly, you can see the whole city! As soon as I settled and friends came to me, the first thing I did was take them to the 25th floor, no joke.

The balconies of only the first 10 floors are visible from the playground. No one is visible above. Children are kicked out of balconies in rare cases, only if caring parents complain to the guards.

It would be convenient if the security guard received a notification about a long movement on a public balcony, or even just saw it on camera and reacted promptly. It is a pity that such technologies have not been invented yet!

Seriously speaking, you can just put a camera that looks at the facade and that’s it. Facade video does not have to be public, security can only be granted access to common balconies.

Conclusion

All this is not some kind of service or a ready-made solution, but only my research fantasy. Video surveillance is not top-down, but bottom-up.

A lot of people don’t want a camera pointed at their balconies, but if asked, do you want to know who dropped a water bag on your car? If there was a fire on the balcony, you were not at home, and the firemen say that this is careless handling of fire, I would like to see who was smoking around this time?

Here the cameras in my LCD seem to be installed well, but if you think about it, their usefulness tends to zero. You can see the silhouette of a criminal who steals a bicycle or how something falls from above and causes damage. Alas, it is almost impossible to find a criminal according to these data, as practice shows. In my house, I would replace half of the cameras with one, but which looks at the facade.

As for me, video surveillance should solve real problems, and not be a useless archive recorder.