Update All Feeds Delay

9 posts / 0 new
Last post
dennisn

It would be nice to have a delay between feed fetches, especially when updating all of them at once, to prevent a single webserver that hosts multiple feeds from getting flooded. Here's an ugly patch that will insert a 4 second delay:

--- requestfeed.cpp 2018-07-01 15:02:45.000000000 -0400
+++ ./src/requestfeed.cpp 2019-03-02 15:47:47.000000000 -0500
@@ -18,6 +18,7 @@
#include "requestfeed.h"
#include "VersionNo.h"
#include "mainapplication.h"
+#include "common.h"

#include
#ifdef HAVE_QT5
@@ -151,6 +152,7 @@
const QDateTime &date, const int &count)
{
qDebug() << objectName() << "::head:" << getUrl.toEncoded() << "feed:" << feedUrl;
+ Common::sleep(4000);
QNetworkRequest request(getUrl);
QString userAgent = QString("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/%1 (KHTML, like Gecko) QuiteRSS/%2 Safari/%1").
arg(qWebKitVersion()).arg(STRPRODUCTVER);
@@ -176,6 +178,7 @@
const QDateTime &date, const int &count)
{
qDebug() << objectName() << "::get:" << getUrl.toEncoded() << "feed:" << feedUrl;
+ Common::sleep(4000);
QNetworkRequest request(getUrl);
request.setRawHeader("Accept", "application/atom+xml,application/rss+xml;q=0.9,application/xml;q=0.8,text/xml;q=0.7,*/*;q=0.6");
QString userAgent = QString("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/%1 (KHTML, like Gecko) QuiteRSS/%2 Safari/%1").

Sheldon
Hi,

Hi,

you can define an update frequency for each feed under

Right click on the feed -> Properties (Ctrl+E).

Kind regards,

Sheldon

dennisn
Yes but that doesn't help in

Yes but that doesn't help in the "Update All Feeds Now" use case... that only applies to people who keep their newsreader open, continuously waiting to poll again. And even then, if many of them have the same frequency, or share common multiples of that frequency, this flooding issue will probably still occur.

Sheldon
The approach comes with

The approach comes with limitations but unfortunately there is no better one so far.

By the way! Such a delay would not really reduce the load.

dennisn
Why wouldn't it reduce load

Why wouldn't it reduce load on the servers - ie. waiting several seconds before sending another request?

Sheldon
Because the load depends on

Because the load depends on the servers' responses too.

dennisn
I'm not sure what you mean.

I'm not sure what you mean. Obviously overall the total amount of data being processed and transferred will be the same, but having it spread out over time is better than requesting everything all at once, surely? (I was referring to the load on the remote servers, btw.)

Sheldon
But you might have

But you might have overlappings depending on the response time, data volume and server traffic too.

The solution from above is the best we can provide so far.

dennisn
Well "might have overlappings

Well "probably won't' have overlappings" is better than "might have overlappings". Currently QuiteRSS polls all (or most of) my youtube feeds simultaneously when starting up the program and updating-all.