Morphex's blogologue (Life, technology, music, politics, business, mental health and more)

This is the blog of Morten W. Petersen, aka. morphex in various places. I blog about my life, and what I find interesting and/or important. This is a personal blog without any editor or a lot of oversight so treat it as such. :)

My email is

I am

An OGG/Vorbis player, implemented in Javascript.

My Kiva bragging page
My shared (open source) code on GitHub

Morphex's Blogodex

Me on Instagram

Zope hosting by Nidelven IT

Morten Petersen on Linkedin

Morten Petersen on Facebook

Morten Petersen on SoundCloud

Morten Petersen on MixCloud

Blogologue on Twitter

Older entries

Atom - Subscribe - Categories

Facebook icon Share on Facebook Google+ icon Share on Google+ Twitter icon Share on Twitter LinkedIn icon Share on LinkedIn

An eventful coding week for Simple TCP proxy

I've been mailing a bit on the python-users mailing list about STP this week, and there were some discussions there that were unfruitful - but I got some useful tips in the process:

This TCP proxy needs to scale and perform reasonably well, so the last thing I worked on was making sure that downloading large files via the proxy works reasonably well, in these commits:

In was interesting, and relieving to see that I could speed up the download process 3-4x by multiplying the "transfer buffer" by 8. A local download of the test.tar file was around 1.3 GB/s via lighttpd, and 122 MB/s via STP. After the buffer change, the speed increased to 449 MB/s. Which is more than what a gigabit connection to the internet can handle.

So, so far so good, and good enough.

The previous commit:

I got a tip for, and reduced the thread stack size to the minimum, which is from what I gather, necessary to ensure that it will run reliably on different platforms - as Linux has a rather big thread size and that could break on others if you're reliant on it.

I also implemented a "fall through", so that the thread sending data back and forth doesn't stop and wait for .N number of seconds, if data has just been sent back and forth.

And in the commit before that, I adjusted the sleep time for threads "polling" on whether or not they have become active:

which also increased the speed at which STP could deal with the connection queue.

There has been some questions on the Python list as to exactly what I'm building and why, but I think a fair description would be that it is a simple to understand, and run, TCP proxy which does some useful things out of the box.

Coming from the web development and hosting world, I'm tempted to add some HTTP protocol features as well, but that's something that belongs in another project, maybe using STP as the frame which it plugs into.

[Permalink] [By morphex] [A simple TCP Proxy (Atom feed)] [31 Jul 20:08 Europe/Oslo]

A simple TCP Proxy

So as I was working on the SMPS, simple message passing system, I looked at ways of protecting it on the public net.

Port knocking was one thing I looked at, and although it looked fine, I thought this was an opportunity to hone my Python/programming skills, and freshen up a bit on programming.

So I ended up writing a simple TCP proxy, and a working copy is here:

The script will setup a proxy in front of a local webserver.

The STP seems a bit sluggish when a lot of connections are made, but I'll have a look at how to optimize that performance.

[Permalink] [By morphex] [An SSL TCP client/server implementation in Python (Atom feed)] [26 Jul 23:52 Europe/Oslo]

Encountering Python crash on numerous client connects to socketserver

On the SMPS package, I've refactored a bit since the last blog post, and I also encountered what looks like a bug in Python.

Python can randomly crash if too many client connections are made to the socketserver, and it crashes in such a way that it is difficult for me as a regular Python programmer to understand what the problem could be.

So although I was tempted to do a bit of C debugging and poke around, I do have limited time and energy, so I did the smart thing and patched it up so that it is easy to reproduce the problem:

and then I created an issue in the Python bug tracker:

[Added later:] Aha. So it turns out I was quitting threads, and this gave an unclear error message. Thanks to Jeff Epler for pointing that out.

That got worked on in this commit:

and the parent commit. Now with > 400 test connections I get an SSL handshake timeout, but, that's in ssl.c and probably due to some configuration setting somewhere.

[Permalink] [By morphex] [An SSL TCP client/server implementation in Python (Atom feed)] [24 Jul 18:13 Europe/Oslo]

Thread-safe message database for SMPS

I hacked a little more on the SMPS project today, which resulted in the following commit:

A pretty straight-forward message passing queue, that expires messages after n number of seconds. I guess the only thing left now before it can be put into practical use, is adding some sort of handshake with a secret, so that it is possible to protect the message queue from unauthorized access.

[Permalink] [By morphex] [An SSL TCP client/server implementation in Python (Atom feed)] [18 Jul 17:37 Europe/Oslo]

An SSL TCP client/server implementation in Python

Related to work I found myself needing a solution to pass messages, so I started working on something simple, just to get the basics down.

I started out with the idea of doing it over HTTPS, but after a bit, I decided to go for a more minimalistic solution, using just TCP over SSL.

I've implemented a simple example of how to implement an SSL protected client/server message passing solution in Python 3.9. I googled a bit, and couldn't find a good, solid, simple example, so I figured the first iteration of the Simple Message Passing System could serve as an example, for others as well.

The committed state of the files [edit: for that first iteration are] here:

[Permalink] [By morphex] [Python and web (Atom feed)] [17 Jul 20:33 Europe/Oslo]