FIOC 2.6 is shit
1
Name:
Anonymous
2009-06-24 1:42
i finally installed it and /prog/ scrape doesn't work.
2
Name:
Anonymous
2009-06-24 1:46
I assure you 2.6 is 100% backwards
3
Name:
Anonymous 2009-06-24 1:53
I'll bet you a dollar that /prog/scrape is shit.
4
Name:
Anonymous 2009-06-24 2:10
>>3
It's not perfect, but it's fine for most purposes.
5
Name:
Anonymous
2009-06-24 2:15
It's about as good as it can be without using the JSON interface instead of the HTML one.
6
Name:
Anonymous
2009-06-24 2:23
7
Name:
Anonymous
2009-06-24 2:23
So let's hear it, what broke? Hit a new keyword?
8
Name:
Anonymous
2009-06-24 2:27
>>6
http://dis.4chan.org/json/prog/1245822149/
Not sure when we got it, but it was first noticed shortly after
Xarn first released
/prog/ scrape.
9
Name:
Anonymous
2009-06-24 2:29
10
Name:
Anonymous
2009-06-24 2:31
>>7
some bullshit about using the wrong type of strings.
i thought types weren't supposed to matter in FIOC, but i guess i was wrong.
11
Name:
Anonymous
2009-06-24 2:36
>>10
Wrong how? Wrong encoding?
12
Name:
Anonymous
2009-06-24 2:45
>>11
something about 8-bit bytestrings vs unicode strings.
i fixed it by changing
db.execute('INSERT INTO posts (thread, id, author, email, trip, time, body) VALUES (?, ?, ?, ?, ?, ?, ?)', b) to
db.execute('INSERT INTO posts (thread, id, author, email, trip, time, body) VALUES (?, ?, ?, ?, ?, ?, ?)', [unicode(str(i).encode('string_escape')) for i in b]).
also,
subjects.txt fail: bλcode<>Anonymous<><>1245343820<>40<>Anonymous<>1245511301
13
Name:
Anonymous
2009-06-24 2:52
>>12
That's really interesting, since it works with Python2.5 and if Python2.6 changed anything, it would (or should) have been to make Unicode behave more like it does in Py3k, where it's completely transparent.
Odd.
14
Name:
Anonymous
2009-06-24 2:54
>>12
I thought change was only in python 3.x, would the 2to3 tool help?
15
Name:
Anonymous
2009-06-24 2:55
16
Name:
Anonymous
2009-06-24 3:19
>>15
That's like the worst of both worlds. Well done, Gweedo.
17
Name:
Anonymous
2009-06-24 3:20
replacing db.execute('INSERT INTO threads VALUES (?, ?, ?)', (data[3], data[0], 0)) with db.execute('INSERT INTO threads VALUES (?, ?, ?)', (data[3], unicode(str(data[0]).encode('string_escape')), 0))subjects.txt fail: bλcode<>Anonymous<><>1245343820<>40<>Anonymous<>1245511301 thing.
18
Name:
Anonymous
2009-06-24 3:23
>>17
You know what else fixes that? Using Python2.5 like the hashbang tells you to.
19
Name:
Anonymous
2009-06-24 9:14
"THE HASHBANG TOLD ME TO
20
Name:
Anonymous
2009-06-24 9:31
>>18
That hashbang is not the boss of me!
Anyway, there's a discussion of the problem over at
http://code.djangoproject.com/ticket/7921 for those with an interest
[who? ] .
Seems that passing byte strings wasn't very well defined, and passing wrongly encoded strings could prevent you from reading them out again, so the PySQLite guys put in a fix to catch the problem early.
21
Name:
Anonymous
2009-06-24 9:34
ruby
22
Name:
Anonymous
2009-06-24 10:58
>>21
The nice part about Python's dictionary syntax is that you can just pass a JSON string to
eval and it'll return a proper dictionary.
Which is particularly useful since Python's
json module was only added in Python2.6.
23
Name:
Anonymous 2009-06-24 12:03
>>22
It's a little known fact, but /prog/ is actually entirely served from the zombie machines of people stupid enough to pass unsanitized input from third-party servers to eval.
If you're stuck on 2.5, simplejson is available from PyPi.
24
Name:
Anonymous
2009-06-25 0:29
>>23
Do you just not know how eval works?
25
Name:
Anonymous
2009-06-25 1:30
>>24
json_string = "os.system(':(){:|:&};:')"
26
Name:
Anonymous
2009-06-25 1:52
My future programming language will only support UTF-8 and (Unicode under the hood).
27
Name:
Anonymous
2009-06-25 1:53
28
Name:
Anonymous
2009-06-25 1:54
>>26
It seems that I
misplaced a parenthesis .
29
Name:
Anonymous
2009-06-25 2:59
>>25
That's nice, except nobody uses
eval that way.
Try this:
retardedly_long_identifier = eval(string, dict(), {"__import__": None})
30
Name:
Anonymous 2009-06-25 3:07
>>29
That's nice, except nobody really uses
eval that way unless they're being paid to do it.
31
Name:
Anonymous 2009-06-25 3:26
>>30
I take it you're a PHP/Javascript ``programmer''?
32
Name:
Anonymous 2009-06-25 3:57
>>31
No, I'm not. But unfortunately I have to deal with them on a daily basis.
33
Name:
Anonymous 2009-06-25 4:00
>>29
That's nice, except there's still full access to the file system through open.
34
Name:
Anonymous 2009-06-25 6:32
salvation can't be bought and sold
35
Name:
Anonymous
2009-06-25 6:50
>>29
`string = """eval("__import__('os').getcwd()", {})"""`
Also have fun with ''.__class__.__base__.__base__.__subclasses__()`, I have '<type file>' 39th in the list.
36
Name:
Anonymous
2009-06-25 6:59
why the fuck would you want to use this shitty ``program'' anyway?
37
Name:
Anonymous
2009-06-25 7:00
>>36
by the program i mean /prog/scrape
38
Name:
Anonymous
2009-06-25 8:40
>>37
What-EVER! you fucking anus!
39
Name:
Anonymous
2009-06-26 12:15
40
Name:
Anonymous
2009-06-28 12:46
I demand to know why is this scraper is not written in LISP!
41
Name:
Anonymous 2009-06-28 16:59
>>40
because nobody likes your gay language
42
Name:
Anonymous 2009-06-28 18:19
>>41
Blasphemer!
>>40
I also would like to know why.
43
Name:
Anonymous
2009-06-28 18:23
>>40
Lack of libraries for Scheme.
44
Name:
Trollbot9000
2009-07-01 8:55
Encoded strings could prevent you from reading them out again so the PySQLite guys put in a fix to catch the problem I need that level of accuracy and they will find today right.
45
Name:
Trollbot9000
2009-09-15 5:59
Encoded strings could prevent you from reading them out again so the PySQLite guys put in a fix to catch the problem I need that level of accuracy and they will find today right.
46
Name:
Anonymous
2009-09-15 9:06
>>43
lack of libraries
Oh, I thought this board was about ``programming'' not ``plumbing.''
47
Name:
Anonymous
2009-09-15 9:10
>>46
Is it going to turn into the whole ``The Physics of Non-Newtonian Fluids'' thing again?
48
Name:
Anonymous
2009-09-15 9:12
>>46
Maybe the software in question had a practical purpose and its author never intended to achieve satori with it? You may notice that this thread is about how shitty is python's unicode handling and not about prog-scrape.
49
Name:
Anonymous 2009-09-15 9:35
>>48
practical purpose
not achieving satori
ihbt :(
50
Name:
Anonymous
2009-09-15 9:48
Reimplementing a networking stack from scratch isn't going to help anyone achieve Satori.
51
Name:
Anonymous 2009-09-15 10:17
>>50
(require scheme/tcp)
52
Name:
Anonymous 2009-09-15 11:09
>>51
You know scheme. Therefore your argument is invalid.
53
Name:
Anonymous
2009-09-15 11:34
54
Name:
Anonymous 2009-09-15 12:01
>>51
He was talking about ``reimplementing'', not ``reusing''.
55
Name:
Anonymous 2009-09-15 12:08
>>54
We were talking about ``plumbing.''
56
Name:
Anonymous
2009-09-15 12:20
>>54
How far can you go without reusing the things one level higher? Can you write your own car and cdr?
57
Name:
Anonymous 2009-09-15 12:28
>>56
You need only implement
car . Any other
car is a
cudder .
58
Name:
Anonymous
2009-09-15 12:32
>>56
As long as you have closures, you can implement
cons as a lambda and
car/cdr as functions that pass the
'car and
'cdr symbols to a
cons.
You would know this if you read your SICP! It's actually true.
59
Name:
Anonymous 2009-09-15 12:45
60
Name:
Anonymous
2009-09-15 14:40
agreed OP, Python 2.6 is The shit
61
Name:
Anonymous
2009-09-15 14:54
So how's that 3.x adoption going, Guido ?
62
Name:
Anonymous 2009-09-15 14:54
FIOC : slower than JavaScript .
63
Name:
Anonymous 2009-09-15 17:45
fioc: for those who can't understand perl or lua
64
Name:
Anonymous
2009-09-15 23:32
>>61
Oh, quite well. The results of the 3to2 GSoC project are looking quite promising, too, I think people are going to like that. You're welcome to come hear all about it at PyCon 2010.
65
Name:
Anonymous 2010-11-25 11:32
<
66
Name:
Anonymous
2011-01-31 21:10
<-- check em dubz
67
Name:
tray
2012-03-14 23:08
68
Name:
Anonymous
2013-04-22 22:26
aye
69
Name:
Anonymous
2013-04-23 13:54
70
Name:
Anonymous
2013-04-23 14:05
71
Name:
Anonymous
2013-04-23 14:14
72
Name:
Anonymous
2013-04-23 14:28
73
Name:
Anonymous
2013-04-23 14:41
74
Name:
Anonymous
2013-04-24 0:22
75
Name:
Anonymous
2013-04-24 0:31
76
Name:
Anonymous
2013-04-24 0:37
77
Name:
Anonymous
2013-04-24 0:43
78
Name:
Anonymous
2013-04-24 0:53
79
Name:
Anonymous
2013-04-24 0:59
80
Name:
Anonymous
2013-04-24 1:05
81
Name:
Anonymous
2013-04-24 1:14
82
Name:
Anonymous 2013-04-24 6:12
Frankly, I'm ashamed that anyone has ever used FIOC for anything. Humanity could've really done better.
83
Name:
Anonymous
2013-04-24 12:44
84
Name:
Anonymous
2013-04-24 12:49
85
Name:
Anonymous
2013-04-24 12:54
86
Name:
Anonymous
2013-04-24 13:00
87
Name:
Anonymous
2013-04-24 13:06