NULL character, cause of

Got a LiveCode personal license? Are you a beginner, hobbyist or educator that's new to LiveCode? This forum is the place to go for help getting started. Welcome!

Moderators: heatherlaine, Klaus, FourthWorld, kevinmiller

Post Reply
montymay
Posts: 106
Joined: Thu Jul 18, 2013 5:23 am

NULL character, cause of

Post by montymay » Wed Dec 27, 2017 5:42 am

I hope there is a simple explanation for some behavior that I do not understand. I am developing on the Windows platform. My script puts a text field with some interior text marked with braces into a variable and parses it with the following code:

Code: Select all

   
     put the htmltext of fld "text" into tText
     put offset("{", tText) into tLeftBracePos
     put char 1 to tLeftBracePos-2 of tText into tText1
     
The script goes on to replace the text inside the braces with "FN" and finally gets the text from the right brace to the end of the text. Then the following statement put the text all back together:

Code: Select all

put tText1 & " FN " & tText2 into tNewNext
The script then writes tNewText to a text file with which I populate a text field, but there is an unwanted empty first line in the text field. When I open the text file with Notepad++, there is a null character in the first line. When I open it with Notepad, there is this: "<p>&#0;</p>", but sometimes I see just Chinese characters! I don't want an empty first line. What is causing this result or how to I prevent it or how to I fix it? Thanks for any suggestions.

Monty May

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 5424
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: NULL character, cause of

Post by dunbarx » Wed Dec 27, 2017 5:56 am

Hi,

No idea what is in tText2, if that matters.

No empty line in the output though. On a Mac, though, and the output placed into an LC field. LC 8.1.8,

Craig Newman

montymay
Posts: 106
Joined: Thu Jul 18, 2013 5:23 am

Re: NULL character, cause of

Post by montymay » Wed Dec 27, 2017 8:21 am

Thanks for your reply. tText2 is the variable that contains all the text that follows the right brace character, which is used to mark the end of what is intended to be a footnote, from there to the last character in the text contained in the tText2 variable. As far as a I know, and I know very little about the details, the content of tText2 is not responsible for creating the empty first line. This behavior may be peculiar to Windows and doesn't happen on the Mac platform. I am thinking it has something to done with the code for line endings on Windows.

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 2598
Joined: Fri Feb 19, 2010 10:17 am
Location: Bulgaria
Contact:

Re: NULL character, cause of

Post by richmond62 » Wed Dec 27, 2017 11:16 am

This is probably a daft idea, but how about putting EMPTY
into your textfield before you populate it with your processed text?

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1694
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: NULL character, cause of

Post by [-hh] » Wed Dec 27, 2017 11:41 am

You could try (if tLeftBracePos>0)

Code: Select all

put char 1 to tLeftBracePos-1 of tText into tText1
Also, for writing to an UTF-8 encoded file, you may need:

Code: Select all

put textEncode(fld "out","UTF-8") into url ("file:" & /path/to/fileout.txt)
shiftLock happens

MaxV
Posts: 1506
Joined: Tue May 28, 2013 2:20 pm
Location: Italy
Contact:

Re: NULL character, cause of

Post by MaxV » Wed Dec 27, 2017 4:16 pm

You are working with htmltext, not with simple text, so please post an example to see what is inside it.
Livecode Wiki: http://livecode.wikia.com
My blog: http://livecodeitalia.blogspot.it
To post code use this: http://tinyurl.com/ogp6d5w

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 2598
Joined: Fri Feb 19, 2010 10:17 am
Location: Bulgaria
Contact:

Re: NULL character, cause of

Post by richmond62 » Thu Dec 28, 2017 10:45 am

I am interested as to why you headed this "NULL".

montymay
Posts: 106
Joined: Thu Jul 18, 2013 5:23 am

Re: NULL character, cause of

Post by montymay » Fri Dec 29, 2017 11:53 pm

Hello MaxV, Richmond62, -hh, and others

In response to Richmond62, I choose the word "NULL" as the description of the topic because when I populate a text field with the file that my script creates, it has an empty first line. When I open the file in Notepad++, it represents the empty line with a NULL symbol:
Image
When I open a file created by my script in Notepad, I see this:
Image

Regarding MaxV's reply, here is the entire script:

Code: Select all

## Assumption: the user has entered some text and marked footnote into field "text" by enclosing it with braces. This script will update the file so that the text appears with a footnote reference 
## and the citation appears in a separate text file listing all the footnotes.

on mouseup
     put empty into tNewText
     local tText, tSkip = "0", tCount = "1", tFNCount = "1", tStart, tEnd
     put the cTitle of me into tTitle
     put "c:/Users/"&gComputer&"/Documents/DB/articles/" into tFolder  
     
     ## (1) get the newly edited text that will be update the article
     put the htmltext of fld "text" into tText
     put tText into fld "text1"
     
     
     ## (2) get text after the new entry and its marked citation & put it into tText2
     put offset("}", tText) into tRightBracePos
     put char tRightBracePos+1 to -1 of tText into tText2
     put tText2 into fld "text2"
     
     ## (3) get just the old text plus the user's entry in the body, but not the citation, and put it into tText1
     put the htmltext of fld "text" into tText
     put offset("{", tText) into tLeftBracePos
     put char 1 to tLeftBracePos-2 of tText into tText1
     put tText1 into fld "text1"
     
     ## (3) get the footnote and put it into tCitation and delete it the marked citation from tFile1
     put char (tLeftBracePos+1) to (tRightBracePos-1) of tText into tCitation
     put tCitation into fld "text3"
     
     ## (4) Put it all together
     put tText1 & " FN " & tText2 into tNewText
     put tNewText into fld "text1"
     
     
     ## (5) get the number of footnote references in the body
     put FNnum("FN", tNewText) into tFNnum
     
     ## (6) Format the new unformatted footnote reference in the variable tNewText while updating the footnote
     ## reference numbers
     set the caseSensitive to "true" 
     repeat tFNnum
          get WordOffset("FN", tNewText, tSkip)  --word
          add it to tSkip  -- put word number of FN 
          if "</sup>" is not in word tSkip of tNewText then  --format & update one-word FN
               put "<a><sup shift="&quote&"-12"&quote&">FN"&tCount& "</sup></a>" into word tSkip of tNewText
               put tCount & ") " & tCitation into tCitation
               add 1 to tSkip  --now three words
          else put "<a><sup shift=" &quote&"-12"&quote& ">FN"&tCount& "</sup></a>" into word (tSkip-1) to tSkip of tNewText  --update three-word FN
          add 1 to tCount
     end repeat
     
     ## (7) Update the body of the text file and display it in the field "text"
     
     open file tFolder&tTitle&".txt" for write
     write tNewText to file tFolder&tTitle&".txt" at 0
     close file tFolder&tTitle&".txt"
     set the HTMLtext of fld "text" to it
     
     ##(8) Update the footnote file
    [script omitted]     
     
     ## (9) Diplay updated file in fld "text"
     [script omiited]     
end mouseup

   function FNnum pSearch, pCont
     put 0 into rFNnum
     put 0 into tCharsToSkip
     repeat
          put offset(pSearch, pCont, tCharsToSkip) into tHit
          if tHit = 0 then exit repeat
          add tHit to tCharsToSkip
          add 1 to rFNnum
     end repeat
     return rFNnum
end FNnum
The resulting field appears as follows:
Image

I will continue studying the problem, following -hh's lead about encoding the text. My thanks to all.

Monty
Attachments
mary.JPG
Capture.JPG
Capture.JPG (20.44 KiB) Viewed 1105 times
Untitled-1.jpg
Untitled-1.jpg (98.16 KiB) Viewed 1105 times

montymay
Posts: 106
Joined: Thu Jul 18, 2013 5:23 am

Re: NULL character, cause of

Post by montymay » Tue Jan 02, 2018 10:13 am

Perhaps I have found the cause of the empty first line in the way I wrote the write to file command. My script read:

Code: Select all

     open file tFolder&tTitle&".txt" for write
     write tNewText to file tFolder&tTitle&".txt" at 0
     close file tFolder&tTitle&".txt"
     set the HTMLtext of fld "text" to it
When I deleted the "at 0" and simply stated "write tNewText to file tFolder&tTitle&".txt", the field populated as I wanted it! I have using 0" because "1" didn't work. The LC dictionary states: "The start specifies the character or byte position in the file where you want to begin writing. A positive number begins start characters after the beginning of the file." The entry for "after" states: "[w[hen you use the after keyword, the current contents of the container . . . is not affected." So I am guessing that "0" is an illegal number and interpreted as a non-negative number. SInce the file was empty before it was written to, it wote to the second line. I'm just hobbyist programmer and so could be wrong, so please correct this entry so that I do not lead a reader astray.

Monty

MaxV
Posts: 1506
Joined: Tue May 28, 2013 2:20 pm
Location: Italy
Contact:

Re: NULL character, cause of

Post by MaxV » Tue Jan 02, 2018 5:11 pm

You don't need anymore to open, just use

Code: Select all

put URL "file:..." into myVar
for reading txt files.

And

Code: Select all

put myVar into  URL "file:..." 
for writing.

Morevoer your code would be much simpler this way:

Code: Select all

replace "{" with ("<a><sup shift="&quote&"-12"&quote&">FN§") in tText
replace "}" with "</sup></a>" in tText
put 0 into tCount
repeat for each char tChar in  tText
if tChar = "§" then 
    add 1 to tCount
    put tCount after tText2
  else
    put tChar after tText2
  end if
end repeat
tText2 contains the new htmlText.
Livecode Wiki: http://livecode.wikia.com
My blog: http://livecodeitalia.blogspot.it
To post code use this: http://tinyurl.com/ogp6d5w

montymay
Posts: 106
Joined: Thu Jul 18, 2013 5:23 am

Re: NULL character, cause of

Post by montymay » Wed Jan 03, 2018 8:47 pm

Thanks, MaxV, for your post. Just two points. I was aware of the URL method but used the alternative because the URL method did not work for me when I used the following syntax:

Code: Select all

put URL "file:"&tFolder&tTitle&".txt" into fld "text"
After many tweaks the solution for me was putting the URL into parentheses! The Dictionary entry for "URL" does not say anything about the need for parentheses.

Code: Select all

put URL ("file:"&tFolderPath&tTitle&".txt") into fld "text"
Two, your script for creating a footnote reference is much simpler and shorter, and I want to use it now, but it does not increment the footnote references that follow the new statement and its footnote reference. I am working on a synthesis of my former method and yours, but if you see a solution, it would be much appreciated.

Monty

MaxV
Posts: 1506
Joined: Tue May 28, 2013 2:20 pm
Location: Italy
Contact:

Re: NULL character, cause of

Post by MaxV » Thu Jan 04, 2018 2:42 pm

Probably you want this:

########CODE to copy and paste#######
on MouseUp
put the htmlText of field "text" into tText
#let's archive all footnotes
put tText into tText2
put 0 into tCount
repeat while matchtext(tText2,"(\{.*\})",tCite)
add 1 to tCount
put offset(tCite,tText2) into tskip
put char (tSkip + the number of chars of tCite + 1) to -1 of tText2 into tText2
put char 2 to -2 of tCite into tCite
put "<a name=FN" & tCount & " ><b>FN" & tCount & "</b>: " & tCite & "</a><br>" & return after tNotes
end repeat
#now we remove them
put replaceText(tText,"\{.*\}", "{}") into tText
#now we add the links
replace "{" with ("<sup><a href=#FN§>FN§") in tText
replace "}" with "</a></sup>" in tText
put 0 into tCount
put empty into tText2
repeat for each char tChar in tText
if tChar = "§" then
add 1 to tCount
put tCount after tText2
else
put tChar after tText2
end if
end repeat
put tText2 & tNotes into URL "file:myTest.html"
launch URL "myTest.html"
end MouseUp
#####END OF CODE generated by this livecode app: http://tinyurl.com/j8xf3xq with livecode 9.0.0-dp-11#####
Livecode Wiki: http://livecode.wikia.com
My blog: http://livecodeitalia.blogspot.it
To post code use this: http://tinyurl.com/ogp6d5w

Thierry
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 583
Joined: Wed Nov 22, 2006 3:42 pm
Location: France
Contact:

Re: NULL character, cause of

Post by Thierry » Fri Jan 05, 2018 8:31 am

montymay wrote:
Wed Jan 03, 2018 8:47 pm
...., but if you see a solution, it would be much appreciated.
Hi Monty,

my take on this:

On this screenshot; top-left is the input text,
bottom-left is the generated output and right the output in my browser.

sunnY- 2018-01-05 à 08.15.52.png
and here is the code:

Code: Select all

function buildFootNotes Txt
   local  Notes, fnCounter, fnText
   local fnLinkTemplate =  "<sup><a href=#FN[[fnCounter]]>FN[[fnCounter]]</a></sup> "
   local fnTextTemplate = "<a name=FN[[fnCounter]]><b>FN[[fnCounter]]</b>: [[fnText]]</a><br>"
   put "<br /><br />_____  notes:  _____<br />" into Notes
   repeat while matchChunk( Txt, "(?x) ( { \s* (.*?) \s* }  )", p1, p2, p3, p4) 
      add 1 to fnCounter
      put char p3 to p4 of Txt into fnText
      put merge( fnLinkTemplate) into char p1 to p2 of Txt
      put  merge( fnTextTemplate)  after Notes
   end repeat
   return Txt  & Notes
end buildFootNotes
and the script button:

Code: Select all

on mouseUp
   local myHtmlFile =  "file:/Users/xxxx/testBuildFootnotes.html"
   put the htmlText of field "fInput" into tText
   set the text of  field "fResult" to  buildFootNotes( tText)
   put  the text of  field "fResult" into URL myHtmlFile
   launch URL myHtmlFile
end mouseUp
HTH,

Thierry
Thierry Douez - https://sunny-tdz.com
Pourquoi tant de notes lorsqu'il suffit de jouer les plus belles... [Barbara]

Post Reply

Return to “Getting Started with LiveCode - Complete Beginners”