[parser] find a better solution than pulling the entire wiki contents into memory
Metadata
Current evaluation
No evaluation has been recorded for this issue yet.
Issue body
Facundo Batista recommends something along these lines for the part processing
def splitgen(text, sep):
pos_from = 0
while True:
try:
pos_to = text.index(sep, pos_from)
except ValueError:
yield text[pos_from:]
break
yield text[pos_from:pos_to]
pos_from = pos_to + len(sep)
test = [
'foo',
'foo\n---\nbar',
'foo\n---\n',
'foo\n---\nbar\n---\n',
'foo\n---\n\n---\nbar\n---\n',
]
for t in test:
assert t.split("\n---\n") == list(splitgen(t, "\n---\n"))
Evaluation history
No evaluation history available.