Share on Facebook
Share on Google+
Share on Twitter
Share on LinkedIn
An eventful coding week for Simple TCP proxy
I've been mailing a bit on the python-users mailing list about STP this week, and there were some discussions there that were unfruitful - but I got some useful tips in the process:https://mail.python.org/pipermail/python-list/2022-July/9070...
This TCP proxy needs to scale and perform reasonably well, so the last thing I worked on was making sure that downloading large files via the proxy works reasonably well, in these commits:
https://github.com/morphex/stp/commit/3328a45d5314040abd37fa...
https://github.com/morphex/stp/commit/5e477bf5756e2763db799c...
In was interesting, and relieving to see that I could speed up the download process 3-4x by multiplying the "transfer buffer" by 8. A local download of the test.tar file was around 1.3 GB/s via lighttpd, and 122 MB/s via STP. After the buffer change, the speed increased to 449 MB/s. Which is more than what a gigabit connection to the internet can handle.
So, so far so good, and good enough.
The previous commit:
https://github.com/morphex/stp/commit/5e477bf5756e2763db799c...
I got a tip for, and reduced the thread stack size to the minimum, which is from what I gather, necessary to ensure that it will run reliably on different platforms - as Linux has a rather big thread size and that could break on others if you're reliant on it.
I also implemented a "fall through", so that the thread sending data back and forth doesn't stop and wait for .N number of seconds, if data has just been sent back and forth.
And in the commit before that, I adjusted the sleep time for threads "polling" on whether or not they have become active:
https://github.com/morphex/stp/commit/9910ca8c80e9d150222b68...
which also increased the speed at which STP could deal with the connection queue.
There has been some questions on the Python list as to exactly what I'm building and why, but I think a fair description would be that it is a simple to understand, and run, TCP proxy which does some useful things out of the box.
Coming from the web development and hosting world, I'm tempted to add some HTTP protocol features as well, but that's something that belongs in another project, maybe using STP as the frame which it plugs into.
[Permalink] [By morphex] [A simple TCP Proxy (Atom feed)] [31 Jul 20:08 Europe/Oslo]
A simple TCP Proxy
So as I was working on the SMPS, simple message passing system, I looked at ways of protecting it on the public net.Port knocking was one thing I looked at, and although it looked fine, I thought this was an opportunity to hone my Python/programming skills, and freshen up a bit on programming.
So I ended up writing a simple TCP proxy, and a working copy is here:
https://github.com/morphex/stp/tree/e781e84eee7b510879593d1a...
The test_http.sh script will setup a proxy in front of a local webserver.
The STP seems a bit sluggish when a lot of connections are made, but I'll have a look at how to optimize that performance.
[Permalink] [By morphex] [An SSL TCP client/server implementation in Python (Atom feed)] [26 Jul 23:52 Europe/Oslo]