Arc Forumnew | comments | leaders | submitlogin
Textile for Arc 0.1
3 points by dpkendal 5194 days ago | 3 comments
I've written briefly about Textile here before. I think I've got it to a releasable stage now, though there's still lots to do. The goal is to provide a more complete implementation of Textile than pg's implementation of Markdown. Here's a very beta 0.1 release: <https://gist.github.com/815476>

The implementation, right now, can do Textile documents featuring just these basic features:

- blocks -- p is the default, can handle classes, styles and langs attached to them but not ids, for now. Extended blocks are supported.

- spans -- all of the default ones are supported, except subscript because I realised that the function wasn't being called because its name contains a tilde, which is a special symbol in Arc. Will be fixed later. no span attributes either for now.

But it's missing the things listed at the top of the file. (there's no escaping of html chars in the normal text, so you can fall back to plain HTML for everything else.) There's still quite a lot to do, but it's coming along nicely, I think.

Some questions for Arc-ers:

1. Is there a better way to do `str-split` than the one I use here?

2. Is it possible to use a callback for a regular expression replacement, like PHP's preg_replace_callback? Or some equivalent of Perl's e mode for regexps?



3 points by rocketnia 5194 days ago | link

Here's a clickable link: https://gist.github.com/815476 ^_^ They're not clickable in submission bodies.

The fact that you're not escaping HTML characters troubles me. I'd kinda prefer not to risk encountering malevolent JavaScript on a forum, even on a forum where most regulars wouldn't abuse that power.

You're using this code for replacing spans like _em_ and @code@:

  (def txt-span (text st et tag) ; st = start textile; et = end textile -- todo: support span attributes
    (re-replace (string (txt-re-quote st) "(\\S.*?\\S?)" (txt-re-quote et)) text (string "<" tag ">" #\\ 1 "</" tag ">")))
It strikes me that this wouldn't handle nesting well, which may be fine, 'cause these spans are things people almost never need to nest--some exceptions being when attributes are involved or when nesting multiple layers of <sup> and <sub>. To make it a bit less sloppy, I recommend manually incrementing an index through the text, using 'begins to identify start tags and maintaining a stack if necessary, even if it sounds horribly ugly to do it that way. :-p This should also give you a good place to insert attribute-parsing code.

Speaking of spans, I'm not sure why you bother making all the txt-@ global variables. I think all you do is use 'eval to define them and another 'eval to get them back, and you don't even need the second 'eval because you have all the information you need.

   (def txt-spans (text)
     (each (span tag) txt-std-spans*
  -    (= text (eval `(,(sym (string "txt-" span)) ,text)))) text)
  +    (zap txt-span text span span tag))
  +  text)
Also speaking of spans, I'm troubled by the fact that there's a -del- span at all. I hope this syntax doesn't apply to every instance of a hyphen before a whitespace character, 'cause that'll mess up a lot of variable names and links.

I realize you're in the earlier stages of getting to know Arc, and I don't mean to discourage you or anything. I only mean to contribute. ^_^

---

Is there a better way to do `str-split` than the one I use here?

The suggestion that comes to my mind is this, but it's kinda laughable....

  (def str-split (blob delim)
    (unless blob (err "Please don't apply 'str-split to nil."))
    (aand blob
          (subst "%1" "%" it)
          (subst "%2" "#" it)
          (subst "#" delim it)
          (map [aand _
                     (subst "#" "%2" it)
                     (subst "%" "%1" it)]
               (tokens it #\#))))
It's basically the same as your current strategy, but instead of letting any one character like #\u0001 slip through the cracks, I do tons more string construction.

The least hackish way is probably to manually iterate using an index.

---

Is it possible to use a callback for a regular expression replacement, like PHP's preg_replace_callback?

Yep, I think so. Anarki's 're-replace depends on Racket's 'regexp-replace* :

  (def re-replace (pat text replacement)
    ($.regexp-replace* ($.pregexp pat) text replacement))
Racket, in turn, has extensive documentation for 'regexp-replace* and the replace-only-the-first-thing variant, 'regexp-replace: (http://docs.racket-lang.org/reference/regexp.html#(def._((qu...)

Apparently you can just keep using 're-replace but pass a function as the "replacement" argument. It appears to work very much like preg_replace_callback(), except that each captured string is passed as a separate argument rather than in an array:

  arc> (load "lib/re.arc")
  nil
  arc> (re-replace "(.).(.)" "abcdefg" (fn (whole x y) (+ y x whole " ")))
  "caabc fddef g"

-----

1 point by rocketnia 5193 days ago | link

I hope this syntax doesn't apply to every instance of a hyphen before a whitespace character

Oops, I meant to say "a hyphen before a non-whitespace character."

Speaking of which, lots of links have +pluses+ in them as URL-encoded spaces, so that's a troublesome syntax too. What does Textile do in these cases?

-----

4 points by dpkendal 5193 days ago | link

> The fact that you're not escaping HTML characters troubles me. I'd kinda prefer not to risk encountering malevolent JavaScript on a forum, even on a forum where most regulars wouldn't abuse that power.

The original classTextile.php (http://code.google.com/p/textpattern/source/browse/developme... -- warning: big) provides a 'restricted' mode, designed for forum comments etc., in which all input <, > and & characters are escaped, which is what I intend to do here as I continue to work on it.

> It strikes me that this wouldn't handle nesting well, which may be fine, 'cause these spans are things people almost never need to nest--some exceptions being when attributes are involved or when nesting multiple layers of <sup> and <sub>. To make it a bit less sloppy, I recommend manually incrementing an index through the text, using 'begins to identify start tags and maintaining a stack if necessary, even if it sounds horribly ugly to do it that way. :-p This should also give you a good place to insert attribute-parsing code.

While you're right about the attribute-parsing, I'm not too concerned about nesting issues because the current reference implementation (there's a dingus at http://textile.sitemonks.com/) doesn't handle that either. It uses the regexp method too (see classTextile.php), with a callback for parsing attributes.

> Speaking of spans, I'm not sure why you bother making all the txt-@ global variables. I think all you do is use 'eval to define them and another 'eval to get them back, and you don't even need the second 'eval because you have all the information you need.

Thanks. I've now corrected my copy and I'll use your method in the next version.

> Also speaking of spans, I'm troubled by the fact that there's a -del- span at all. I hope this syntax doesn't apply to every instance of a hyphen before a whitespace character, 'cause that'll mess up a lot of variable names and links.

> Speaking of which, lots of links have +pluses+ in them as URL-encoded spaces, so that's a troublesome syntax too. What does Textile do in these cases?

Good point, that's a special case I missed. The function is now:

    (def txt-span (text st et tag) ; st = start textile; et = end textile -- todo: support span attributes
      (re-replace (string "(?<=\\W|^)" (txt-re-quote st) "(\\S.*?\\S?)" (txt-re-quote et) "(?=\\W|$)") text (string "<" tag ">" #\\ 1 "</" tag ">")))
(notice the lookarounds at each end) which should prevent such issues.

> I realize you're in the earlier stages of getting to know Arc, and I don't mean to discourage you or anything. I only mean to contribute. ^_^

Yes! Thank you for your contribution -- it's for advice like this that I released so early.

> The suggestion that comes to my mind is this, but it's kinda laughable....

Yeah, I'll stick with my `str-split` for now. It should be a library function anyhow, so I'd rather keep my version short and simple until there's a version in `strings.arc` or `arc.arc`.

> Apparently you can just keep using 're-replace but pass a function as the "replacement" argument. It appears to work very much like preg_replace_callback(), except that each captured string is passed as a separate argument rather than in an array

Aha, I don't know why I didn't think to try that. I just mentally assumed I would need another function to do it.

Thanks for all your help! I really appreciate it.

(There's now a textile repo at https://github.com/dpkendal/textile, into which I'll slowly be putting improvements.)

-----