Edit Sup Issue: issue8

Summary
Issue type Status
Sup version Ruby version
Assigned To Tags (list)
Description
Attachments
  Make a copy

Created on 2009-11-10.06:59:42 by rlane, last changed by gauteh.

Files
File name Uploaded Type Edit Remove
0001-test-if-encoding-is-valid.patch gauteh, 2009-12-18.17:47:05 text/x-patch edit
mbox-binary-encoding.patch rlane, 2009-11-10.06:59:42 text/x-patch edit
Messages
msg98 (view) Author: gauteh Date: 2009-12-25.14:03:30
I think somebody mentioned this earlier, but when pressing 'q' to exit nothing 
happens when pressing 'y'. The only way to exit is by doing 'Q'.

- gaute
msg96 (view) Author: gauteh Date: 2009-12-25.13:49:36
I pulled your encoding branch and rebased it on your master, I got this when 
trying to restore my index (10000+ messages). Same as earlier.

[sup.git](rl_enc*) $ bin/sup-sync --restored --restore ~/sup-index-20091218b 
Loading state dump from /home/gaute/sup-index-20091218b...
Read 10719 entries from dump file.
Scanning maildir:///home/gaute/.mail/gaute.vetsj.com/Gmail.All_Mail...
/home/gaute/dev/ruby/sup.git/lib/sup/message-chunks.rb:112:in `initialize': 
undefined method `valid_encoding?' for nil:NilClass (NoMethodError)
	from /home/gaute/dev/ruby/sup.git/lib/sup/message.rb:539:in `new'
	from /home/gaute/dev/ruby/sup.git/lib/sup/message.rb:539:in 
`message_to_chunks'
	from /home/gaute/dev/ruby/sup.git/lib/sup/message.rb:479:in `block in 
message_to_chunks'
	from /home/gaute/dev/ruby/sup.git/lib/sup/message.rb:479:in `map'
	from /home/gaute/dev/ruby/sup.git/lib/sup/message.rb:479:in 
`message_to_chunks'
	from /home/gaute/dev/ruby/sup.git/lib/sup/message.rb:247:in 
`load_from_source!'
	from /home/gaute/dev/ruby/sup.git/lib/sup/message.rb:343:in 
`build_from_source'
	from /home/gaute/dev/ruby/sup.git/lib/sup/poll.rb:149:in `block in 
each_message_from'
	from /home/gaute/dev/ruby/sup.git/lib/sup/maildir.rb:161:in `block in 
each'
	from /home/gaute/dev/ruby/sup.git/lib/sup/maildir.rb:158:in `upto'
	from /home/gaute/dev/ruby/sup.git/lib/sup/maildir.rb:158:in `each'
	from /home/gaute/dev/ruby/sup.git/lib/sup/util.rb:586:in `__pass'
	from /home/gaute/dev/ruby/sup.git/lib/sup/util.rb:573:in 
`method_missing'
	from /home/gaute/dev/ruby/sup.git/lib/sup/poll.rb:143:in 
`each_message_from'
	from /home/gaute/dev/ruby/sup.git/lib/sup/util.rb:546:in 
`method_missing'
	from bin/sup-sync:147:in `block in <main>'
	from bin/sup-sync:142:in `each'
	from bin/sup-sync:142:in `<main>'

when doing:
diff --git a/lib/sup/message-chunks.rb b/lib/sup/message-chunks.rb
index edc37a3..6ef3d66 100644
--- a/lib/sup/message-chunks.rb
+++ b/lib/sup/message-chunks.rb
@@ -109,7 +109,9 @@ EOS
       end
 
       @lines = nil
-      if text
+      if text and not text.valid_encoding?
+          print "Invalid encoding", text
+      elsif text
         @lines = text.gsub("\r\n", "\n").gsub(/\t/, "        ").gsub(/\r/, 
"").split("\n")
         @lines = lines.map {|l| l.chomp.wrap WRAP_LEN}.flatten
         @quotable = true

I see that it fails on the same GIF-image as earlier (I only assume its a GIF 
since the header prints GIF89a something). Why it thinks it is text is a mystery 
to me, but then again I don't know the internals of sup very good. This is the 
only place it fails..

The subjects are working here.
msg92 (view) Author: rlane Date: 2009-12-19.05:08:34
And by "anonymous" I mean "rlane". I should really give the email interface to
this thing a shot...
msg91 (view) Author: anonymous Date: 2009-12-19.05:04:54
The "encoding" branch at my github should work better. I modified
Iconv.easy_decode to asciify text it was unable to transcode. 

Commit 4bc5fae53 fixes the missing subject issue.
msg90 (view) Author: gauteh Date: 2009-12-18.18:22:52
Unsure if it is releated, but none of the restored messages have subjects.
msg89 (view) Author: gauteh Date: 2009-12-18.17:47:05
i've been testing out your 1.9-binary branch a bit. im getting this when trying 
to --restore my entire index:

[~] $ sup-sync --restored --restore sup-index-20091218b 
Loading state dump from sup-index-20091218b...
Read 10719 entries from dump file.
Scanning maildir:///home/gaute/.mail/gaute.vetsj.com/Gmail.All_Mail...
/home/gaute/.gem/ruby/1.9.1/gems/sup-999/lib/sup/message-chunks.rb:113:in 
`gsub': invalid byte sequence in UTF-8 (ArgumentError)
	from /home/gaute/.gem/ruby/1.9.1/gems/sup-999/lib/sup/message-
chunks.rb:113:in `initialize'
	from /home/gaute/.gem/ruby/1.9.1/gems/sup-999/lib/sup/message.rb:537:in 
`new'
	from /home/gaute/.gem/ruby/1.9.1/gems/sup-999/lib/sup/message.rb:537:in 
`message_to_chunks'

[... snip. ]

When printing out the text that fails it seems to be a GIF image.. likely some 
spam. Testing if the encoding is valid gets it through.. but some errors are 
spit out (from Iconv??):
[sup.git](19*) $ ruby bin/sup-sync --restored --restore /home/gaute/sup-index-
20091218b 
Loading state dump from /home/gaute/sup-index-20091218b...
Read 10719 entries from dump file.
Scanning maildir:///home/gaute/.mail/gaute.vetsj.com/Gmail.All_Mail...
[2009-12-18 18:42:47 +0100] WARNING: couldn't transcode text from unknown-8bit 
to UTF-8 ("Kan du sjå desse: Ã"...) (got invalid encoding ("UTF-8", "unknown-
8bit"))
[2009-12-18 18:42:49 +0100] WARNING: couldn't transcode text from utf-8 to UTF-8 
("32΂̂¨΂³ñÁėö"...) (got invalid byte sequence in UTF-8)
[2009-12-18 18:43:01 +0100] WARNING: couldn't transcode text from utf-8 to UTF-8 
("ˑR̃[åώ¸ç"...) (got invalid byte sequence in UTF-8)
Scanned 10394, added 0, updated 0 messages from 
maildir:///home/gaute/.mail/gaute.vetsj.com/Gmail.All_Mail.
Scanning maildir:///home/gaute/.mail/stud.hib.no/INBOX/...
Scanned 328, added 0, updated 0 messages from 
maildir:///home/gaute/.mail/stud.hib.no/INBOX/.
Scanning sup://drafts...
Scanned 0, added 0, updated 0 messages from sup://drafts.
Scanning sup://sent...
Scanned 0, added 0, updated 0 messages from sup://sent.

- gaute
msg8 (view) Author: rlane Date: 2009-11-10.06:59:42
Tracking bug for 1.9 compatibility.

ncurses needs a few fixes: http://github.com/rlane/ncurses

rmail has encoding issues: http://github.com/rlane/rubymail

Opening mbox files with the "BINARY" encoding seems to work. Patch attached.


The rmail patch above is very hacky. But, these combined are enough to let me
index a bunch of spam and run sup with no obvious problems.
History
Date User Action Args
2009-12-25 14:03:30gautehsetmessages: + msg98
2009-12-25 13:49:36gautehsetmessages: + msg96
2009-12-19 05:08:34rlanesetmessages: + msg92
2009-12-19 05:04:54anonymoussetmessages: + msg91
2009-12-18 18:22:52gautehsetmessages: + msg90
2009-12-18 17:47:05gautehsetfiles: + 0001-test-if-encoding-is-valid.patch
status: unread -> chatting
messages: + msg89
2009-11-10 06:59:42rlanecreate