(Server Config) Slow Upload with Direct Server Uploads

ollie

Member
YetiShare User
Apr 25, 2013
75
5
8
United Kingdom
Hi All, i think Adam has probably had enough with my tickets so i am going to post here from now on, hopefully someone can help me.

I'm using a dedicated file server with direct uploading but i'm having problems achieving good upload speeds through the website. I am able to achieve over 30MB/s uploading to the file server over FTP. However through the website i can only get 3MB/s.

I've attached my php and nginx config, if anyone has any ideas as to why this is happening please let me know.

I've tried increasing the chunk size but it made no difference

Server specs:

2x Quad Xeon E5620 (16 Cores)
64 GB RAM
120GB SSD 8x 4 TB SATA RAID 10
1Gbps

Thanks in advance,
Ollie
 

Attachments

Last edited:

adam

Administrator
Staff member
Dec 5, 2009
2,043
108
63
Hi Ollie,

Is your site on https? If yes ftp is non secure so won't have the overhead of encryption. Might be worth testing upload speed via scp.

Also are you using apache or nginx? The web server will handle the upload itself, then when done its passed to php, so I'd say the web server is the place to look. Does it pause after each chunk or not really?

Thanks,
Adam.
 

adam

Administrator
Staff member
Dec 5, 2009
2,043
108
63
Sorry I see you're using Nginx. I'll see if there's any info about nginx upload speed issues...
 

adam

Administrator
Staff member
Dec 5, 2009
2,043
108
63
That nginx config also tells me you're using ssl (sorry its late.. :oops:). Maybe test the scp upload speed as mentioned above.
 

ollie

Member
YetiShare User
Apr 25, 2013
75
5
8
United Kingdom
Hi Adam, thank you for the reply, just tested uploading to the file server and got 70MB/s using scp

Downloading/Uploading videos for conversion to the conversion server also goes very slowly (same server as the above test) I also tried a html5 upload test to file server and was able to achieve full speed.

Thanks
Ollie
 
Last edited:

ollie

Member
YetiShare User
Apr 25, 2013
75
5
8
United Kingdom
Tested uploading with this php upload script and was able to get full speed also tried a different file server on yetishare and got full speed so i don't know anymore :(
 

adam

Administrator
Staff member
Dec 5, 2009
2,043
108
63
@ollie I'll run some tests locally with that script and get back to you with the results.
 
  • Like
Reactions: ollie

adam

Administrator
Staff member
Dec 5, 2009
2,043
108
63
That suggests the chunk size is only 2MB. Can you check the upload_max_filesize & post_max_size on the main server? If it's set to 2MB that'll cause the chunk size to be 2MB. You should see it chunk at 100MB.
 

ollie

Member
YetiShare User
Apr 25, 2013
75
5
8
United Kingdom
Thanks Adam, didn't realise the main server needed it too. Weird thing is that the speed was fine when i tried a different file server, oh well won't question it thanks for the help!
 

adam

Administrator
Staff member
Dec 5, 2009
2,043
108
63
Strange, did that resolve the upload speed issue?
 

vertigo

New Member
Jun 11, 2022
2
0
0
Hi Ollie,

Know this post is old but can you confirm the upload speed you achieved and outline how you achieved it.

We have not yet cracked the 2mb/s even with the setting noted.

We are using b2 cloud storage and I know the server only has 100mb connection, but think we should get higher upload speeds.

thx