Escapes in string literals

[mwieder]

And your leadership in writing "doubled double-quote" is appreciate; that is clear.)

English is such a weird language.

[DarScott]

You two have almost convinced me to cheer allowing single quote bracketed string literals everywhere and allowing the other quote inside.

However, I'd like to see other bids for the single quote and apostrophe. That is, what is the opportunity cost?

As far as multiple lines... I make stack libraries for folks. When I need large text constants I put them in properties of the library stack or fields on design cards of the library stack--however, I have seen a script that gets them from the comments of the script itself which might be handy for controls or button scripts. (For example, I just put some JavaScript into some library stack properties.) When it is just a few lines, I use multiple get or multiple put after. I see nothing wrong with a continued line, its just that I'm from a functional programming background and I had to beat out of me the tendency to write all functions as a single large return command on many lines. I should ease up on that.

[monte]

Hmm.... would backtick be a better option to use interchangeably with double quote. Single quote is more likely to have an uneven number in your average text and the idea of adding an apostrophe and then having to change the chars surrounding the string seems annoying to me. Either way we need something to replace \" in format/match so I'd rather that be consistent everywhere rather than invent \q for that one case.

[LCMark]

@monte:

Hmm.... would backtick be a better option to use interchangeably with double quote. Single quote is more likely to have an uneven number in your average text and the idea of adding an apostrophe and then having to change the chars surrounding the string seems annoying to me.

Oh, please not backtick! The character is awkward to type (its one of the ones that moves around a lot on keyboard layouts) and difficult to read (and always makes me think something more is going on - e.g. as in bash scripts!).

Either way we need something to replace \" in format/match so I'd rather that be consistent everywhere rather than invent \q for that one case.

I think revisiting the idea that strings remain as-is and we just encourage people to use 'format()' to do text containing returns and quotes might be a good idea. I see no issue with introducing '\q' as a replacement for '\n' in format() since it is very much consistent with the idea of C-style string formatting.

I must confess I don't like doubled double-quote very much, nor doubled single-quote for escaping quotes (in the appropriate context) as it isn't particularly readable (but perhaps a bit more readable than " & quote & "?).

Ultimately there are limits to what we can do here...

The 'format()' (and matchText()) functions need to change as they are an abomination in terms of language-design for no real benefit (requiring syntax parsing and lexical analysis to be interleaved) and that can be done in an automated fashion as part of 'Clean Syntax' and the script translater. This, therefore, would be an ideal time to introduce '\q' (to replace '\"').

Adding doubled double-quote could be down now with no forward or backward compatibility issues (I'm not saying that is enough justification for doing it, however!).

I guess I have nothing in principal against allowing single-quotes or double-quotes to delimit a string although it only partially solves the problem (what if you want a string with both " and ' in it?) and has the issue (just like HTML does) of 'which should I use by default'. However, we have the (small) problem then that single-quote is actually part of the structure of identifiers. e.g. tVar's is actually a valid identifier, which means even a script translator couldn't do 100% accurate job due to dynamicity and 'do'. [ That being said, cutting down on the 'allowed chars' in identifiers was something on the list to do for the 'Clean Syntax' project - it seems a suitable time to take the hit and, to be fair, the number of scripts out there using single-quote in identifiers is probably next to none... ].

Of course, there's always the option of making C-style formatting the default in string literals (another change that could be made at the 'Clean Syntax' switch-over). However, this seems wrong to me somehow - it's imposing more knowledge on the scripter for the most basic of things something which seems to go against the grain of what the LiveCode language is meant to be about.

So, having said all of that how about this for another suggestion:

Code: Select all

  put @"This is a c-style formatted string allowing quotes \q, returns \n and tabs \t"

This is forward and backward-compatible (i.e. could be done now) - means that you pay-for-what-you-use conceptually and is a lot less heavy than having to use 'format()' to achieve the same effect. We could even consider extending the escaping to allow variable interpolation like perl:

Code: Select all

  put @"The value of tVar is \(tVar)"

[DarScott]

The @ would address the original concern. It doesn't seem very LiveCode to me. Scripts are read, not only written, and this might be confusing for some.

If it is used, then @ should work hard. It might have the general abstract or even vague notion of switching to some cryptic notation, the first implemented is the special string notation. The reader might think "oh, that symbol, no wonder this makes no sense, I need to look up at-quote."

However, we already have an @ for parameter declarations and it has no meaning related to the meaning of special string literals.

Because "at" is a preposition, @ feels like a binary operator to me or command modifier. But, it already has history of a prefix.

[mwieder]

Adding doubled double-quote could be down now with no forward or backward compatibility issues (I'm not saying that is enough justification for doing it, however!).

Not quite true. Implementing doubling at this point would commit this to the syntax, since it would create backward compatibility problems in the future if we ever decided to back down and people had taken advantage of the opportunity to use that syntax.

tVar's is actually a valid identifier

OMG. I know Mats Wilstrand has experimented with using punctuation as variable names and I reprimanded him for it, but I don't know of any other actual use of this "feature". I wouldn't be at all distraught about closing this loophole.

[mwieder]

put @"The value of tVar is \(tVar)"

*I* like this, but as you point out, it's not very xtalk at all. And as Dar points out, we already have a use for the @ symbol for variable passing, and I dislike reusing operators for different purposes. Maybe

Code: Select all

      put eval("The value of tVar is \(tVar)")

would work better.

[LCMark]

Not quite true. Implementing doubling at this point would commit this to the syntax, since it would create backward compatibility problems in the future if we ever decided to back down and people had taken advantage of the opportunity to use that syntax.

Well backwards-compatible meaning it wouldn't affect existing scripts, and forward-compatible in that its compatible with the new syntax parser and its requirements... Any addition that is made to syntax commits us for the future whether it be string-literal changes or additions/alterations to commands and functions. (Although, admittedly, changes at the lexical level are slightly more critical since it can't be changed by Open Language - the latter means syntax will be quite easily mutable but people will still expect a strong, consistent and supported core).

OMG. I know Mats Wilstrand has experimented with using punctuation as variable names and I reprimanded him for it, but I don't know of any other actual use of this "feature". I wouldn't be at all distraught about closing this loophole.

Heh - '.' also falls into the category of things that are allowed in identifiers, as does '?' if I recall correctly. RunRev has (since Revolution 1.0) always recommended people stick to the normal (perhaps slightly boring!) [_a-zA-Z][0-9_a-zA-Z]* structure to identifiers to guard against future potential incompatibilities.

[LCMark]

*I* like this, but as you point out, it's not very xtalk at all. And as Dar points out, we already have a use for the @ symbol for variable passing, and I dislike reusing operators for different purposes.

I had forgotten about the use of '@' for reference parameters when I wrote that I must confess (I had just been looking at Obj-C - which uses @"..." to indicate an NSString literal). Although I think it is perhaps the only symbol that could be used for the purpose (it's about the only one left that even remotely works well as a prefix and even then - as Dar pointed out - not that well).

In terms of 'eval("The value of tVar is \(tVar)")', we already have 'merge()' which works like that essentially albeit without escapes.

Ultimately, if we want a more convenient way to specify string literals which can contain things that current strings don't and 'format("...")' is considered too 'inconvenient' then either some non-xtalkish symbols are going to have to creep in, or some other English-like mechanism is needed (like a 'escapeStrings' as a context-local property).

Of course, this is where we need someone to step in and present some perfect syntactic solution that pleases everyone all of the time

[mwieder]

Of course, this is where we need someone to step in and present some perfect syntactic solution that pleases everyone all of the time

...and that's why you get the big bucks...

Last week I sat in on a linguistics lecture at Heidelberg University on word morphology where we explored eight different points of view of what defines a word.

[monte]

I don't particularly like "" either but I do like consistency which \q doesn't allow for because as previously stated that can't ever be supported in string literals outside format/match...

[LCMark]

I'm not sure I see the issue with '\q' as an alternative to '\"' in escaped strings regardless of whether we introduce doubled double-quote or not - its entirely logical and consistent with c-style escaping after all. Also, there are options for supporting escaped strings outside of format/match (e.g. escapedStrings context-local property).

Just to sum up the suggestions so far (for enabling quotes, and in some cases other characters in strings):

• DoubleQuotes: Doubled double-quote - e.g. "Quote - "" - see".
• EitherQuotes: Single or double-quote - e.g. 'Quote - " - see', "Quote - ' - see".
• EscapeProp: Context local property (e.g. escapedStrings) - set the escapedStrings to true - "Quote - \q - see".
• Format: Just make people use format - format("Quote - \q - see")
• Symbol: Symbol prefix (e.g. @) - @"Quote - \q - see"

To my mind, here are the pros and cons...

DoubleQuotes
+ Backwards-compatible and no issues moving forward either (beyond taste!).
+ Allows expressing of strings containing both double-quote or single-quote.
- Cannot express strings consisting of other characters that could be useful (e.g. return, tab).
- Difficult to read quickly, potentially error prone (both of these would probably be significantly mitigated by syntax highlighting).

EitherQuotes
+ Consistency with some other languages (including HTML).
+ Allows expressing of strings containing either double-quote or single-quote.
- Cannot express strings containing both double-quote and single-quote
- Cannot express strings consisting of other characters that could be useful (e.g. return, tab).
- Results in two almost identical ways of doing the same thing - i.e. "string" or 'string' - reduces script consistency.

EscapeProp
+ Backwards-compatible and no issues moving forward.
+ Entirely optional in terms of use.
+ Allows escaping of all characters that could be useful.
+ Could potentially allow variable interpolation as well.
+ Pay-for-what-you-use conceptually.
- Cannot determine what a string means by just reading the line it is on - you need to check context for the setting of 'escapeStrings'.

Format
+ No changes required.
+ Entirely optional in terms of use.
+ Allows escaping of all characters that could be useful.
+ Pay-for-what-you-use conceptually.
- Slightly more to type for each string constant you want to use escapes in.

Symbol
+ Backwards-compatible and no issues moving forward.
+ Entirely optional in terms of use.
+ Allows escaping of all characters that could be useful.
+ Pay-for-what-you-use conceptually.
+ Escaped strings require only one extra character and its effect is entirely local to that string.
- Not very xtalk like

Note: Don't take the number of +/-'s in the above lists to indicate how good each suggestion is - I'm just trying to lay out the current options which have been proposed for improving the expressibility of string constants thus far in the discussion.

[mwieder]

@runrevmark: thanks for the summation.
I'd also like to point out that this isn't a winner-take-all situation... the options aren't necessarily mutually exclusive.
So while I'm still a proponent of the "EitherQuotes" solution, it doesn't (for example) rule out escaping chars. Allowing both would, I think, eliminate most of the drawbacks of either.

EitherQuotes:

- Results in two almost identical ways of doing the same thing - i.e. "string" or 'string' - reduces script consistency.
But it allows for more flexibility in formatting SQL statements, etc.
Also, I don't see why this is less consistent than allowing case-insensitivity in variable names.

[LCMark]

I'd also like to point out that this isn't a winner-take-all situation... the options aren't necessarily mutually exclusive.

Indeed they aren't

Also, I don't see why this is less consistent than allowing case-insensitivity in variable names.

I guess in my mind it is different - but then I treat keywords and variables as if they were case-sensitive almost exclusively having worked in languages which are case-sensitive prior to LiveCode.

[mwieder]

I do the same. Force of habit, I guess. I get annoyed when I see references where the case doesn't match.