Morphex's blogologue (Life, technology, music, politics, business, mental health and more)

Taking a look at my Python surveil(lance) app

So, I created this surveillance app in Python, to surveil (https://github.com/morphex/surveil) the room where I spend most of my time, just to make sure that nobody else visits it, without my approval.

Before I wrote this app, I saw there were different applications out there, that could do some sort of surveillance, but I guess I recognized early on that I could easily mail images to myself, and that this was a good approach as it kind of disconnects the surveil app from outside dependencies, at least it doesn't have to have an internet connection up absolutely all the time, to function.

Another feature of mailing myself images compiled into videos, is that as soon as it comes into (in my case) GMail's system, there is a record of the video, and it is because of that, difficult to manipulate data, when a mail has been delivered.

Python was the language of choice, because I wanted to make things easy for myself, and Python is the language I've worked with the most, and it is easy to read and write things in Python.

Before this, I had dabbled a bit with ffmpeg, playing around with videos, adding effects to them and so on.

I'd say fortunately I dabbled with ffmpeg, because it is quite a powerful video processing package. The command line is not intuitive and user friendly, but once the command line is right, ffmpeg seems to be pretty stable.

I read about Python and the GIL (Global Interpreter Lock) today, and I remember this from years back. I guess over 90% of the programming work I've done to date, is in Python and Zope 2. Zope 2 had a way around the GIL and exploiting all CPU cores, and that was running a main database server, and then having several database clients/applications on each of their on process, which effectively went around the GIL, as each application was one process to the operating system. This worked well as the system was built for few writes to the database and many reads.

So, fortunately I dabbled with ffmpeg before this project, because I soon realized that threading could be a bit of a headache, so I opted for running ffmpeg as a subprocess of the surveil app, and using shell scripting and files to pass messages; so when ffmpeg is done processing a set of images into a video, it creates a file, and a thread with a loop in the surveil app monitors for these files, then mails the video and deletes the files related to that video afterwards.

Python is simple and intuitive, and that's the main reason I think, that it became my programming language of choice. It was easy to get started, interesting to explore Python via the interpreter, and there was an advanced web system which used Python as the main programming language.

Years back, I was pretty much obsessed with cross-platform portability, but with this surveil app, I'm creating a temporary (RAM-based) file system which is Linux-specific, and I'd be happy if it runs on *nix/POSIX systems. It's all in keeping things simple.

So I guess the point of this post was to underline that, yes, Python has a GIL, but in most instances it is a non-issue, and if there are CPU-intensive things, these can be forked out to a command-line tool, or you could even have an extension in C which does the heavy lifting. If I hadn't learnt ffmpeg, I could easily have spent a lot more time writing this app.

Another point that is worth mentioning, is the config.py file for this project, is easy to read, and goes with the Python style of simplicity, and goes one step beyond being a config file, it also has some (easy to understand) logic.

This project is a mix of different systems; Python to glue it all together, *nix file systems and scripting in a middle layer, and the ffmpeg application for the heavy-duty work. As far as I can tell, this is all good and stable, although the surveil app reboots about every 24 hours, so it is hard to know how well it runs with a 100% uptime.

[Permalink] [By morphex] [A surveillance app (Python tree here I come!) (Atom feed)] [23 Oct 15:26 Europe/Oslo]