The question is, how can I change the default text qualifier in Web Scraper, from ", to say, a backtick (`). This would avoid the CSV file breaking, because I have many instances of unmatched double quotes in the text which stand for inches, and must be left as is. Thus with "24"" (24 inches), you can see the boundaries for the text are going to shift - which messes up the entire csv file. You can see that having `24"` solves the problem. But Webscraper doesn't seem to have an option to set your preferred text qualifier?
Hi are you certain WS is the issue? AFAIK it generates valid CSV files. BTW double quotes can be used within CSV fields, but they need to be escaped. So for something like 24" monitor, the CSV field should be " 24"" monitor" (two consecutive double quotes after 24).
You can confirm whether the problem lies with WS by opening its generated CSV files in Notepad. You should see fields like " 24"" monitor" properly escaped. Another way to confirm is to open the CSV file in Excel or Google Sheets. If it renders correctly (you should see 24" monitor, and no broken columns), then the problem lies elsewhere.
Some 3rd-party programs, CSV libraries and custom code do not handle CSV escaping properly so they may render the WS CSV files incorrectly.
1 Like
@leemeng
So what happens when you have an empty field? "" becomes interpreted, or is indistinguishable from, an escaped double quote.