zzz.i2p

Development discussions
Much better and more "consistant" torrent downloads running I2PSnark and BigkyBT at the same time. « Bugs, Complaints, Requests, Questions ... « I2P Help, Hints, Advice
 
Thu, 22 Nov 2018, 10:33pm #1
YesYesYes well maybe
I2P Legend

I have much better and more "consistent" torrent downloads running I2PSnark and BiglyBT at the same time.

There's a bug somewhere. If I run I2PSark alone I get these periods of 12KBps or less frequently. Very frequently. This even with java set to 2GB in wrapper.config. Raising the number of tunnels in I2Psanrk doesn't do the same thing as running both torrent downloaders at the same time so I don't see tunnels as the problem. Lots of data on what I tried here,

http://zzz.i2p/topics/2641-slow-down-error-out-...

Here's where I noted running both I2Psnark and BiglyBT at the same time.

http://zzz.i2p/topics/2623?page=1#p14186

I've also run I2Pd with BiglyBT and while I don't get stellar speeds they don't fall as low. It seems to consistently do medium speeds well.

Something about running I2PSnark alone makes the downloads almost halt at times. A lot. It's also using a vast amount of memory. I'm running up on and sometimes using all the 2GB allocated for java.exe

I'm not the only one that's noticed this. Here

http://i2pforum.i2p/viewtopic.php?f=12&t=51...

here

http://i2pforum.i2p/viewtopic.php?f=12&t=6&...

Someone that understands and is able to monitor this situation would be much better placed to understand what's going on than I would. There's only so much I can do.

Maybe running both I2Psnark and BiglyBT at the same time would show what the difference is???

I could be wrong but I think it likely that this started showing up after zzz fixed the timer problem that caused I2PSnark to slowly grind down to very low downloads unless you restrated. In this case restarting doesn't help.

trac filed

http://trac.i2p2.i2p/ticket/2357#ticket

Sat, 24 Nov 2018, 12:45pm #2
YesYesYes well maybe
I2P Legend

I notice that like the other person that I linked to

http://i2pforum.i2p/viewtopic.php?f=12&t=51...

that I'm not uploading much at all to anyone else. It's not like my bandwidth is restricting this. I want to upload to others. I can see that there's also lots of seeds or people who have a lot of the files I need and they are uploading to me a minuscule amount. Maybe instead of me having problems downloading it's everyone has problems uploading to others.

P.S. I do appreciate all the work that has gone into I2P. It's not like I want to complain. Something has changed though and it's lagging the whole system.

Sat, 24 Nov 2018, 04:13pm #3
echelon
I2P Legend

Hi

it seems you look for a red herring and put to much effort in hunting ghosts.
I2P is a full dynamic network and you cannot control anything beside your own router.

Current I2Psnark rates are quite fine and running fast and good for me. Far better than 1,2,3 or 4 years ago. With 5 active torrents I get 500 kb/sec or more download max on my limited I2P router.
Sure, each single connection is limited and may not reach 5 kb/sec (while others do reach 40) , but sum of all makes it.

echelon

Sat, 24 Nov 2018, 08:44pm #4
YesYesYes well maybe
I2P Legend

echelon wrote:

Hi

it seems you look for a red herring and put to much effort in hunting ghosts.echelon

My first comment here was Sep 2015. So I've been running I2P and I2PSnark most every day before that date. Maybe I can't give you a direct reason that I believe it's slowed but it seems it has to me and two other people. Surely in this amount of time I've begun to get some sort of feel for the rates that it downloads. This is not just a overnight thing.

echelon wrote:

I2P is a full dynamic network and you cannot control anything beside your own router.echelon

It's insulting for you to tell me these things. I don't know this??? Of course I do. My complaint is for averages over time. Not instantaneous download speeds. Read the comment above. Even if I don't have the technical skills you have...well an analogy, if a person watched cars go by their house attentively for as long as I have watched I2PSnark and they told you the traffic slowed down would you lecture them about the road surface or road building rates or tell them the traffic moved fine by their house?

echelon wrote:

Current I2Psnark rates are quite fine and running fast and good for me. Far better than 1,2,3 or 4 years ago. With 5 active torrents I get 500 kb/sec or more download max on my limited I2P router.
Sure, each single connection is limited and may not reach 5 kb/sec (while others do reach 40) , but sum of all makes it.echelon

I used to get a lot of small ones but but the speeds have dropped to more like 500Bps, or less, and many times the Seeds will not download at all for long periods when they do at all. If I could get 500KBps I would be jumping for joy. I actually got 300KBps and commented that I was gloriously happy but it very soon petered out. I don't know why.

It may very well be that you and zzz have the impression that I've experimented with my install until I screwed something up. Not so. I have all the installed values in a file and have made changes only when it falls off a cliff. I only changed things when it dropped so low that all the seeds were gone because it took so long to download files.

Sun, 25 Nov 2018, 09:32am #5
YesYesYes well maybe
I2P Legend

I found a concrete example I can give you. I've been running I2PSnark and BiglyBT at the same time to see what's going on. BigllyBT has a lot more information than I2PSnark to look at. There's a TV series I've been downloading for months. Veerrryyy ssslllooowwllly. I started using BiglyBT in parallel, (I start I2PSnark, wait til it is fully started before starting Bigly BT and make sure BT writes it's downloads to a different folder.) I made a perfect copy of all my torrent downloads into another folder and then directed BiglyBT downloads to it. Some finished, some a little and some very little. A mix of completion. The BiglyBt TV series torrent is at 98.%% and the I2PSnark is at 85%. I was going to say the BT is not downloading to the Snark, it wasn't, until I started writing this. It is now doing so but the difference in files is quite great. How come it took so long, hours and hours, and possibly days to start downloading the larger BT to the smaller Snark? I have restarted and it ran 88 minutes before it started transferring files to the smaller(Snark) and before I restarted it ran for hours and made no effort to balance the difference. The difference is about 860MB. That's a large difference. Most importantly I can see both servers the BT in Snark and the Snark in BT. So they know of each others existence and in BT I can even see what parts the Snark has yet they will not equalize.(They both could see each other and not equalizing has been over more than one day hence the difference in file size). We're talking slow downloads here of 1.2KBps or so over days for a 8GB file of which I only want the last seasons, maybe 6GB.

I don't know if it matters but I don't want all the files in the torrent. I have the first seasons of the torrent so I canceled the first seasons in both BT and Snark.

Setting up something like this and watching carefully what is going on when this happens might be a clue to what I and others have been complaining about. They and I say that there is no seeding done when there is opportunity to do so. echelon is saying that the little downloads add up. I agree but the little downloads are not happening at all in some cases.

I also lowered the number of tunnels BiglyBT was using in,

http://127.0.0.1:7657/configtunnels

It had 6 Client tunnels for BiglyBT: DHT Pure and I think 3 Client tunnels for BiglyBT: DHT Mixed. I lowered to 3 and 2 respectively.

Sun, 25 Nov 2018, 11:54am #6
echelon
I2P Legend

Hi

Sorry if it seems like I did got over you.

But I still think you look for a red herring.
Comparing 2 different programs is not really useful, as both have different queues, choking methods and at least different tunnels. Not really much usage in comparing both at the same time on the same files, esp. as both will choose different nodes.
Also the net changes every day, the users change every day, active torrents change every day. It is quite normal for rates to vary heavy, because some days more nodes are active, other days only slow nodes are active.
And in snark neraly nothing on bandwidth related changes has happen the last releases. If the net would have been the same, the rate would be the same.
But net changes, the nodes on torrent changed, the users doing torrent changed, the clients used by the torrent users changed.
There is really not much gain in comparing torrent rates from weeks ago with today rates.
More important would be internal i2p things like RTT, window size, nodes participating,...

Sure you can look out of the window and see the cars slowing down, but that could be because of slower cars, different cars, different drivers.
Thats why you do look on these values for some time and see different bandwidth values.
Even the avg change over time is easy to explain: some time ago more torrent nodes maybe online with higher bandwidth allotments. Or faster nodes in I2P in general.

Thats the bad point of the dynamic I2P network: you see the result, but you nearly cannot tell why as to much variable changing to fast.

On the technical part:
- only a few cosmetic changes in I2Pnark has happen AFAIK
- NTCP2 on i2p router has happen
- some small changes and bugfixes in router has happen, which did not result in less bandwidth on observation

echelon

Sun, 25 Nov 2018, 09:52pm #7
YesYesYes well maybe
I2P Legend

Arrgh. My apologies. I may be wrong. The "BT at 98.6% and Snark at 85%" may be because BT counts percentages as what you "wish" to download and Snark counts percentages as what you "actually" download. So if I refuse to download some it would skew the results. I will see tomorrow when I hope they will have finished what I selected to download then I will know for sure. This of course, [might], kill my "concrete case" that downloading is not always taking advantage of opportunities to download files available. I'll have to watch the files that are 100% downloaded and see if they are equal tomorrow. Sigh.

[Yes it was the difference between BT and Snark measured completeness]

Mon, 26 Nov 2018, 08:58pm #8
YesYesYes well maybe
I2P Legend

echelon wrote:

Comparing 2 different programs is not really useful, as both have different queues, choking methods and at least different tunnels. Not really much usage in comparing both at the same time on the same files, esp. as both will choose different nodes"

Not so if you watch average downloads over days and weeks time. BT also gives more information. I like I2PSnark better but BT has more info.

echelon wrote:

Also the net changes every day, the users change every day, active torrents change every day. It is quite normal for rates to vary heavy, because some days more nodes are active, other days only slow nodes are active.
And in snark neraly nothing on bandwidth related changes has happen the last releases. If the net would have been the same, the rate would be the same.
But net changes, the nodes on torrent changed, the users doing torrent changed, the clients used by the torrent users changed.
There is really not much gain in comparing torrent rates from weeks ago with today rates.
More important would be internal i2p things like RTT, window size, nodes participating,...
Thats why you do look on these values for some time and see different bandwidth values.
Even the avg change over time is easy to explain: some time ago more torrent nodes maybe online with higher bandwidth allotments. Or faster nodes in I2P in general.

Thats the bad point of the dynamic I2P network: you see the result, but you nearly cannot tell why as to much variable changing to fast.

On the technical part:
- only a few cosmetic changes in I2Pnark has happen AFAIK
- NTCP2 on i2p router has happen
- some small changes and bugfixes in router has happen, which did not result in less bandwidth on observation

You're not telling me anything I don't already know. I fully understand that it works for you. What you are doing is pretending that either I or the others that have complained and used I2P for a long time are hallucinating or can't see download rates from seeds. I say that somewhere around 0.9.33 or earlier the download rates plummeted by at least 50% or much more and became very sporadic with very brief times of high download bandwidth of say 120-200KBps but mostly hovering much lower at 22-30KBps. Also getting a lot of

Rejecting tunnels: High message delay

which I didn't have (near as frequent)before with a 32 bit operating system with 4GB memory until around 0.9.33 and maybe earlier. Not sure. I now have lots of warnings with 8GB of memory. One of the reasons I upgraded was because of this. To no avail. The reason I know this is because I installed a new OS plus I2P around the same time and that was 0.9.33.

I can see easily that with lots of seeds available to various torrents they barely download. It's been almost a year since 0.9.33. This is not something I've just seen since a few weeks ago. The reasons I haven't said anything was I wanted to try and do all I could to see if it was something on my side so I would not be complaining all the time. I hate to complain. It's really annoying to me to have to. There's really no other way to change things though so I'm complaining.

I've been using I2P since i2pinstall_0.6.4 and most every day since maybe 0.9.24 and most likely every day since much earlier.

I've provided you with two others that have much the same complaints as me,

http://i2pforum.i2p/viewtopic.php?f=12&t=51...

http://i2pforum.i2p/viewtopic.php?f=12&t=6&...

Are we all hallucinating? Don't know how to count?

In the links I posted,

zzz,"...Ultimately, snark is our network load tester..."

I've said about all I can. If you just ignore me then...well I'll just be ignored. I will eventually just get a newsgroup or proxy account.

Mon, 26 Nov 2018, 09:03pm #9
YesYesYes well maybe
I2P Legend

I just thought of something. I keep getting

Rejecting tunnels: High message delay

how do I change the values at which point I reject tunnels? I'd rather have a delayed tunnel than none at all. What is this setting and where is it?

Tue, 27 Nov 2018, 12:49pm #10
echelon
I2P Legend

Hi

message delay is the time a message takes in your system, if it is to high, you do not route others tunnels. Only way to bring it down is less system load, better, faster system.

And yes, you are long time user, but still, all changes every time, as you know. So you cannot compare values from 3 weeks ago with today values, even not on avg. You still compare apple and pears.
So you still hunt for ghosts if you compare those values as a base for your values.

And yes, even if you run lots of seeds, limits, other values, tunnels,... will stop other from download all at once at full speed. But you know this already.

All I can tell: on my firewalled limited i2p router (0.9.37) I get values >300 kb/sec for one torrent with 20 peers in download and >50 kb/sec in upload. And thats far better than 3 years ago while I only got 100-150 kb/sec.
But I got only ~ 10 active torrents in snark.

Maybe you should try with 10 active torrents and see if you reach these values, or they still keep low on 4-5 kb/sec.

echelon

Wed, 28 Nov 2018, 02:51am #11
YesYesYes well maybe
I2P Legend

First you must understand I'm not trying to be rude or troll you. I'm very grateful for the hard work that has been put into I2P by all of you and I have said so many times. Thank you!

echelon wrote:

...And yes, you are long time user, but still, all changes every time, as you know. So you cannot compare values from 3 weeks ago with today values, even not on avg. You still compare apple and pears...

Great thanks but you are still not paying the slightest attention to what I have said. Three weeks???? We're talking almost a year. I've only begun complaining because I see no alternative. So if I compare say 10 months with two years earlier I would say that's a fairly long time to average performance. Like the other poster I haven't complained because I haven't really nailed down exactly what is the problem, except it seems to be slowing down and backing up.

I realize this is awful and thin gruel to study and I don't pretend it's not but it is what it is.

echelon wrote:

...So you still hunt for ghosts if you compare those values as a base for your values.

Exactly how long would I need to see slow downloads to have a serious measure???

echelon wrote:

...All I can tell: on my firewalled limited i2p router (0.9.37) I get values >300 kb/sec for one torrent with 20 peers in download and >50 kb/sec in upload. And thats far better than 3 years ago while I only got 100-150 kb/sec.
But I got only ~ 10 active torrents in snark.

Maybe you should try with 10 active torrents and see if you reach these values, or they still keep low on 4-5 kb/sec.

I haven't gone to 10 torrents, I will try it, but I did go from 82 down to 32 of which I stopped maybe 5, so 27 active. There seemed to be no big difference at all. Maybe they would download a little faster but I never got any bandwidth increase as you would think. The reason I have so many is...they are backing up and taking forever to download. However I have had just as many and more and had decent downloads before the problems started. Back when I had Win XP with only 4GB of memory I've had 110 torrents at once and yes it slowed a little but they all finished in a reasonable time.

As a test I shut off anything that doesn't have a good number of seeds or participating servers to 10 total....I got it down to 11 but 3 have only one seed and I don't want to shut them down. They've been on a long, long time and I don't want to lose the last seed.

I have seen quick reaction in the past to this tactic but not recently. I'll wait 24 hours and see.

Thu, 29 Nov 2018, 02:45pm #12
echelon
I2P Legend

Hi

it is not a matter of time in your data, it is the situation of the net and your connection which are important.
Time is irrelevant.
No use for 2 weeks, 10 weeks or 100 weeks data.
You need the same situation on the same clients with the same tunnels to have really useful data and easy to compareable situation with different i2p versions. (e.g. the same 5 active torrents, the same 30 clients, the same active tunnels with the same paricipants, every week the same)
no matter how long ago a situation is.
But you will never get that, so all is just a wild guessing into the dark, as you cannot describe/control the situation behind your network connection.
And as long as those situation behind your ISP are changing every minute, you get more or less worthless data, even if you collect 10 years.

All you have now is a nice overview of your bandwidth with your torrents. And some guesses about the net, which are really guesses into the wild, until you have data of other i2p members and a situation controlled fully by you.
You can make use of your data if you can quantify the situation of the network for each data collection point you have, e.g. bandwidth per tunnel, state of snark (choke, which torrent active,..) and the situation of the active tunnels (10 clients, enough connection free,..).

echelon

Sat, 01 Dec 2018, 04:54am #13
YesYesYes well maybe
I2P Legend

Edit zzz post removed due to inapproriate language
user banned for 30 days

Last edited: Mon, 03 Dec 2018, 01:09pm by zzz

Sun, 02 Dec 2018, 02:27pm #14
echelon
I2P Legend

Stop being racist, ok?

Correct data collection and correlation those is a hard topic and takes time to do it right.

Every data has a meaning, nothing is useless. But you need to get more than the pure data to have a trustfully, usefull interpretation of that data. And thats what we tell you: you cannot guess without more information than you already got, or it will be a wild guess into the blue, based on your (personal, not hard based on facts) point of interpretation.

And the high message delay is completely independent of the net, it is based on your local router.
It will stop you from routing participating tunnels, but in a side effect, high message delay tells you, your router system is slow (for various reason). This leads to slower bandwidth, as it takes longer to process the packages and RTT increases.
High message delay can be more often with changes of crypto in the last releases, as that crypto processing takes slightly longer than the older one. Maybe your system was on the edge and pushed now a bit more forward into that warning.

echelon

Mon, 03 Dec 2018, 12:02am #15
zab
I2P Legend

YesYes, if you are this determined to improve your snark speeds, I would like you to help me test out a theory as to what is causing the irregular speeds in snark.

The theory is explained here http://trac.i2p2.i2p/ticket/2280

The magnet to download an I2P build with the default pipeline increased from 5 to 7 is magnet:?xt=urn:btih:4ca84e117939e8841bf8e560060017ebeadd9185&dn=Z-Lab+release%3A+More+aggressive+Snark+downloads&tr=http://tracker2.postman.i2p/announce.php

If you wish to build from source, the patch is here http://paste.crypthost.i2p/?476fe5124d227c35#iPDAR2DTvph7SMuZDUvaomUx0DRBDML34Z18lX/F9cM=

Last edited: Mon, 03 Dec 2018, 12:51am by zab


email: zab@mail.i2p Irc2P/keybase: zlatinb
lab: http://z-lab.i2p
blog: http://zab.i2p

Sat, 08 Dec 2018, 08:31pm #16
Qubes
I2P Legend

I haven't used BiglyBT in at least 6 month. When you torrent at TB level i2p is a drop in the bucket. I was wondering if anybody looked at the BiglyBT router (an older version) to see if it does do any additional reseeding, connections, key exchanges etc. I was interested in key exchanges over i2p for clear-net torrents. I was not impressed at the time when I used Bigly but maybe they improved.

Mon, 10 Dec 2018, 05:44pm #17
parg
I2P Legend

What weren't you impressed with?

For things to improve it often helps if the developers are made aware of where improvements might be sought.

Thu, 13 Dec 2018, 03:23am #18
Qubes
I2P Legend

I use aria2 with the java wrapper and qbittorrent (with Vuze blocklist) because of VM memory (I keep torrents on an external loop format HardDrive). Bigly had to many resources dedicated to the program. I also torrent over Tor from time to time when the torrent is only 1GB or less (speeds are 300 to 600 kb/s).

For a massive program like BiglyBT I expected https://wiki.vuze.com/w/Safepeer_Migration

I was also expecting more i2p and tor for trackers and searches... Maybe if it becomes modular where you can turn off features, I will reconsider... After all one of my favorite current VM OS Parrot I considered junked 10 years ago when Palomino was depressed about some hacker betrail..

Thu, 13 Dec 2018, 10:59pm #19
parg
I2P Legend

Safepeer? I was more interested in your comments regarding "additional reseeding, connections, key exchanges etc. I was interested in key exchanges over i2p for clear-net torrents" - what were you looking for here?

Sun, 16 Dec 2018, 04:33pm #20
Qubes
I2P Legend

You can establish a private i2p network (zero length SAM tunnels between Bigly routers). You can pass anything you like untill you piss enough people off to get banned. Or use Aktie Syndie... Tor onion share etc... Yes if you don't have safekeeper you will not succeed.