The last three days or so, syncing on Runbox 7 is quite slow, compared to how it was before.
Larry
the difference in the syncing is because of the change from the link below, which was released 3 days ago. The main difference is that while earlier every change (marking as read, moving to another folder, flagging etc), caused the message to be fully reloaded and reindexed, now only the headers (subject, date, etc), are reloaded and only the changes are reindexed. So you should see a less messages be indexed while, new messages (where contents need to be loaded) may take bit more time, since they need to be parsed for extracting the text content. This change also does the full parsing and preloading of the new message so that it’s ready when you click on it. The parsed version is cached, so this is only once mer message. The previous version where text content were retrieved in full for every change used a different mechanism for only extracting text (see source snippet below), which is faster for the purpose of just extracting text, but less accurate than the full mailparser (GitHub - nodemailer/mailparser: Decode mime formatted e-mails), which is capable of handling much more when it comes to content types and encoding.
So in short. Syncing is now faster for changes, since loading less data, there are also less syncing entries than earlier. It’s slower for new messages that have not been parsed earlier.
We can also improve this by caching the parsed results from the server-side indexer, so that when the user starts Runbox7, text contents for most of the new messages are ready to be served, and should not take any extra time to fetch.
sub extract_msg_plaintxt {
my $self = shift;
my $msgfilename = shift;
open(my $fh, $msgfilename);
my $lastContentTypeIsText = 1;
my $headerArea = 1;
my $boundarySeparator;
my $contentTransferEncoding = "";
while (my $row = <$fh>) {
if($lastContentTypeIsText &&
$headerArea &&
$row=~/^[\n\r]/
) {
$headerArea = 0;
}
if($row=~ /boundary=\"*([^\"]+)/) { #"
$boundarySeparator = $1;
}
if($row=~ /Content-Type: ([a-z\/]+)/) {
if($1 eq 'text/plain') {
$lastContentTypeIsText = 1;
} else {
$lastContentTypeIsText = 0;
}
}
if($row=~ /Content-Transfer-Encoding: ([a-z0-9\-]+)/) {
$contentTransferEncoding = $1;
}
if($headerArea) {
if(
$row=~ /^Message-Id:/i ||
$row=~ /^In-Reply-To:/ ||
$row=~ /^References:/ ||
$row=~ /^From:/ ||
$row=~ /^To:/ ||
$row=~ /^Cc:/ ||
$row=~ /^Subject:/ ||
$row=~ /^Date:/) {
#print $row;
}
} else {
if(
# Stop parsing if we see a boundary separator
$boundarySeparator && index($row,$boundarySeparator) !=-1
) {
$lastContentTypeIsText = 0;
last;
}
if($lastContentTypeIsText) {
if($contentTransferEncoding eq "quoted-printable") {
$row = decode_qp($row);
} elsif($contentTransferEncoding eq "base64") {
$row = decode_base64($row);
}
# trim
$row =~ s/^\s+|\s+$/ /g;
# remove newlines and repeating spaces
$row =~ s/\s+/ /g;
# check if we have more than 10 characters and no spaces
if(length($row)>10 && $row !~ /\s/) {
# line contains no spaces - we don't want this in the search index
last;
}
print $row;
}
}
}
close($fh);
}
1 Like
Thank you for the explanation. I was afraid it might be my internet connection or computer.
I think some work needs to be done with the speed of the email synchronization for the web client. It really takes a long time (at least a full minutes) for my index to finish each time I log in to the web mail client.
Let me add my 5 cts: what is slower is the progression that it shown on the grey bar when loading new messages. It likely is due to what @petersalomonsen describes above. Truth is that the feeling described by @lgregg is real (I use FF on Win10, recent laptop, gigabit connection).
Would it be possible to load just the headers to spare the user the waiting for the bodies, render the lines corresponding to the messages and let the user click on one already while loading the bodies in the background? The body of the clicked message would be loaded (indexed or not) separately from the others.
2 Likes
Yes. That’s my plan for August updates. Will separate indexing of headers and body as you suggest, and also load multiple message bodies in parallell ( most of the delay you experience is waiting for server sending the message body, and it’s possilble to request several message bodies in parallell)
2 Likes
I just turned syncing off. It’s too slow.
Work is in progress: https://github.com/runbox/runbox7/pull/227
Once this is released, should improve a lot (and better than it was earlier). Feel free to test locally if you want.
1 Like
Deployed to production now - so the latest version is now way faster.
4 Likes
I just wanted to say that things are much improved!! Now you can go back to claiming to be the fast email service on the planet. 
5 Likes
Really??? I sure don’t see it. A new message icon shows up by the inbox folder but it takes a minute or more for it to actually show up in the folder! I use the mail on my iphone (imap I think) and it’s way faster than 7 on my computer. It’s my biggest complaint with 7. At times it shows old emails that were deleted but show unread. Overall I don’t like it and since they’re getting rid of Roundcube, I’m not sure I’m going to stay when my subscription comes up unless they do some major fixing.
@Keith, can you provide some details about your system and browser? Also, which Runbox 7 build are you seeing (bottom left corner)?
Does it help to reload the app and/or clear the browser cache?
– Geir
It has Runbox 7 build time: 2019-02-05T11:58:08.894Z.
I’m using WIN 7 with Firefox version 69 64 bit. Reloading seems to clear it but I shouldn’t have to do that. I have the same issues on another computer as well which leads me to believe it’s not a computer issue…
Hi Keith. This “Runbox 7 build time: 2019-02-05T11:58:08.894Z” is a really old version from 6 months ago. The most recent version is “Runbox 7 build time: 2019-09-07T19:41:58.425Z” which is 3 days ago. Could you try the newer version please? In the recent versions it will prompt you to update when there is a new version.
Hi Dave, That’s pretty interesting because I always tell it to update. In fact I just had one the other day but I see now it hasn’t changed. I’m going to try and delete the cookies and see if it changes.
Yes, that does sound unusual. Let us know how you get on in case there is something we need to look at further.
OK after I cleared the cookies, it switched the current version you have. We’ll see how that works.
Well, I’m happy to say that the issues I had seem to be gone now. It is much faster and the icon lag is gone now. I’m going to keep tract of the version change when it asks me to update to make sure it’s really gets applied. Both of my computers are doing that so I wonder if it’s a Firefox thing.
Thanks for the help.