Uploading files through "shaped" connections without traffic control

Technically shaping is limiting the rate at which packets are sent over a connection. If you want to continue surfing the web or interacting with websites while uploading your files to online storage, you have to find some way to shape your connection to your online storage. Not only will you not be able to interact with websites while uploading files, but your transfer statistics won't be accurate.

There are really only two ways you can shape a connection without traffic control. You can either use a relay that supports delaying packets or find some way to slowdown the rate at which your file is sent.

In any case, the first thing you have to do is determine how much of your bandwidth you want to dedicate to uploading files. Then you have to convert your bandwidth to a unit measurement that is recognizable by the command rsync. When you're ready you can play with the next command.

Here's a one-liner for transferring a file from a remote server via sshd then uploading it to online storage via mount.davfs at directory cloud, and limiting the rate at which the file is sent to 450 Kbps:

rsync --bwlimit=55 --progress $(scp -2 -v root@remote_server:/root/dir/$(n=file_name; echo $n) .; echo $n) cloud/dir/$n

Do you have a suggestion about how to improve this blog? Let's talk about it. Contact me at David.Brenner.Jr@Gmail.com or 720-584-5229.

Comments

The meaning of time in reinforcement learning

Reinforcement learning (RL) is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning is concerned with how software agents ought to take actions in an environment in order to maximize the notion of cumulative reward through the process of trial and error. In reinforcement learning an agent starts at an empty state then analyzes the available datasets according to a policy of positive states and negative states. Rather than being explicitly taught as in supervised learning the correct set of actions for performing a task, reinforcement learning uses rewards as signals for positive states and punishments as signals for negative states. The agent obtains the best path to a desirable reward as a cumulation of positive states and negative states. As compared to unsupervised learning, reinforcement learning is different in terms of goals. While the goal in unsupervised learning is to find similarities and differences...

Threat hunting polymorphic malware in Linux with Python

You can investigate suspicious activity that could be polymorphic malware by collecting relevant machine data from your endpoint. You can use the machine data to create your own analysis. Before you start your investigation you will need to determine normal activity on your endpoint. Normal activity is the scope of functionality of the software on your endpoint during periods of low activity and high activity. You will need some kind of software that periodically collects specific machine data from your endpoint like my software developed in Python that's available for free download at https://github.com/davidbrennerjr/server-stats-collector Ingest one or more of the following machine data from Category #1. Ingest one or more of the following machine data from Category #2. And ingest one or more of the following machine data from Category #3. Category #1 General system-wide error messages from /var/log/syslog Auditing logs of application rule...

Application behavior monitoring in Linux with Python

You can monitor application behaviors by collecting relevant machine data from your endpoint. You can use the machine data to investigate suspicious activity and create your own analysis. Before you start your investigation you will need to determine normal activity on your endpoint. Normal activity is the scope of functionality of the software on your endpoint during periods of low activity and high activity. You will need some kind of software that periodically collects specific machine data from your endpoint like my software developed in Python that's available for free download at https://github.com/davidbrennerjr/server-stats-collector Ingest one or more of the following machine data from Category #1. Ingest one or more of the following machine data from Category #2. Category #1 General system-wide error messages from /var/log/syslog Auditing logs of application rulesets Auditing logs of security contexts Auditing logs of ...

David Brenner Jr

Search This Blog