Extended ASCII and non-standard characters a problem?

I've seen Crystal reports will throw a "hex" error on bad characters, there is an answer-book on Epicweb for finding and fixing the offending field.

Also ran into a field length problem on a master file once.
Manual keyboard entry on the form would be limited to 25 characters on the form.
However, we discovered you could paste longer strings into the field and it would save without error.
But would then result in an odd processing error downstream... I think that one took an hour to find.

Our sales folks will often copy and past from MS Word directly the Order Entry screen, and many "non-standard" characters end up in the LineDescr field.

Characters like:  â„¢ ® © “ ” (fancy quotes), etc ...


And occasionally  control characters like  the tab character ( \t, \x07, chr(7), etc ...)


Other then possible display issues (we've already had the â„¢ appear as    Ã¢â€žÂ¢   ), can these characters cause DB issues?


FWIW, our DB (OE 10.1b) is UTF-8.


Calvin

 

Oh man, it can affect more than you think. I'm a huge advocate of using just normal alpha numeric. What can special characters mess up? Dashboards, BAQ's, BPM's, Barcoding, and any customs you're trying to do. Reporting can become a nightmare. Keep It Simple (KIS). 

 
Jonathan Lang
I.T Manager
Database Engineer



On Tuesday, February 16, 2016 10:50 AM, "ckrusen1@... [vantage]" <vantage@yahoogroups.com> wrote:


 
<div id="ygrps-yiv-89630860yiv3454079676ygrp-text">
  
  
  <div></div><div><span>Our sales folks will often copy and past from MS Word directly the Order Entry screen, and many &quot;non-standard&quot; characters end up in the LineDescr field.</span></div><div><span>Characters like: &nbsp;™ ® © “ ” (</span><span style="word-spacing:normal;">fancy quotes), etc ...</span></div><div><span style="word-spacing:normal;"><br clear="none"></span></div><div><span style="word-spacing:normal;">And occasionally &nbsp;control characters like &nbsp;the tab character ( &#92;t, &#92;x07, chr(7), etc ...)</span></div><div><span><br clear="none"></span></div><div><span>Other then possible display issues (we&#39;ve already had the&nbsp;™&nbsp;appear as &nbsp; &nbsp;â„¢ &nbsp; ), can these characters cause DB issues?</span></div><div><span><br clear="none"></span></div><div><span>FWIW, our DB (OE 10.1b) is UTF-8.</span></div><div><span><br clear="none"></span></div><div><span>Calvin</span></div><div><span>&nbsp;</span></div>

</div>
 


<div style="color:#fff;height:0;"></div></div>
#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676 -- #ygrps-yiv-89630860yiv3454079676ygrp-mkp { border:1px solid #d8d8d8;font-family:Arial;margin:10px 0;padding:0 10px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mkp hr {
border:1px solid #d8d8d8;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mkp #ygrps-yiv-89630860yiv3454079676hd {
color:#628c2a;font-size:85%;font-weight:700;line-height:122%;margin:10px 0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mkp #ygrps-yiv-89630860yiv3454079676ads {
margin-bottom:10px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mkp .ygrps-yiv-89630860yiv3454079676ad {
padding:0 0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mkp .ygrps-yiv-89630860yiv3454079676ad p {
margin:0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mkp .ygrps-yiv-89630860yiv3454079676ad a {
color:#0000ff;text-decoration:none;}
#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-sponsor #ygrps-yiv-89630860yiv3454079676ygrp-lc {
font-family:Arial;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-sponsor #ygrps-yiv-89630860yiv3454079676ygrp-lc #ygrps-yiv-89630860yiv3454079676hd {
margin:10px 0px;font-weight:700;font-size:78%;line-height:122%;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-sponsor #ygrps-yiv-89630860yiv3454079676ygrp-lc .ygrps-yiv-89630860yiv3454079676ad {
margin-bottom:10px;padding:0 0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676actions {
font-family:Verdana;font-size:11px;padding:10px 0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676activity {
background-color:#e0ecee;float:left;font-family:Verdana;font-size:10px;padding:10px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676activity span {
font-weight:700;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676activity span:first-child {
text-transform:uppercase;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676activity span a {
color:#5085b6;text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676activity span span {
color:#ff7900;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676activity span .ygrps-yiv-89630860yiv3454079676underline {
text-decoration:underline;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676attach {
clear:both;display:table;font-family:Arial;font-size:12px;padding:10px 0;width:400px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676attach div a {
text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676attach img {
border:none;padding-right:5px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676attach label {
display:block;margin-bottom:5px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676attach label a {
text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 blockquote {
margin:0 0 0 4px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676bold {
font-family:Arial;font-size:13px;font-weight:700;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676bold a {
text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 dd.ygrps-yiv-89630860yiv3454079676last p a {
font-family:Verdana;font-weight:700;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 dd.ygrps-yiv-89630860yiv3454079676last p span {
margin-right:10px;font-family:Verdana;font-weight:700;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 dd.ygrps-yiv-89630860yiv3454079676last p span.ygrps-yiv-89630860yiv3454079676yshortcuts {
margin-right:0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676attach-table div div a {
text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676attach-table {
width:400px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676file-title a, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676file-title a:active, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676file-title a:hover, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676file-title a:visited {
text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676photo-title a, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676photo-title a:active, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676photo-title a:hover, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div.ygrps-yiv-89630860yiv3454079676photo-title a:visited {
text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 div#ygrps-yiv-89630860yiv3454079676ygrp-mlmsg #ygrps-yiv-89630860yiv3454079676ygrp-msg p a span.ygrps-yiv-89630860yiv3454079676yshortcuts {
font-family:Verdana;font-size:10px;font-weight:normal;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676green {
color:#628c2a;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676MsoNormal {
margin:0 0 0 0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 o {
font-size:0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676photos div {
float:left;width:72px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676photos div div {
border:1px solid #666666;height:62px;overflow:hidden;width:62px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676photos div label {
color:#666666;font-size:10px;overflow:hidden;text-align:center;white-space:nowrap;width:64px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676reco-category {
font-size:77%;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676reco-desc {
font-size:77%;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 .ygrps-yiv-89630860yiv3454079676replbq {
margin:4px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-actbar div a:first-child {
margin-right:2px;padding-right:5px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mlmsg {
font-size:13px;font-family:Arial, helvetica, clean, sans-serif;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mlmsg table {
font-size:inherit;font:100%;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mlmsg select, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 input, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 textarea {
font:99% Arial, Helvetica, clean, sans-serif;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mlmsg pre, #ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 code {
font:115% monospace;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mlmsg * {
line-height:1.22em;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-mlmsg #ygrps-yiv-89630860yiv3454079676logo {
padding-bottom:10px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-msg p a {
font-family:Verdana;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-msg p#ygrps-yiv-89630860yiv3454079676attach-count span {
color:#1E66AE;font-weight:700;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-reco #ygrps-yiv-89630860yiv3454079676reco-head {
color:#ff7900;font-weight:700;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-reco {
margin-bottom:20px;padding:0px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-sponsor #ygrps-yiv-89630860yiv3454079676ov li a {
font-size:130%;text-decoration:none;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-sponsor #ygrps-yiv-89630860yiv3454079676ov li {
font-size:77%;list-style-type:square;padding:6px 0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-sponsor #ygrps-yiv-89630860yiv3454079676ov ul {
margin:0;padding:0 0 0 8px;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-text {
font-family:Georgia;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-text p {
margin:0 0 1em 0;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-text tt {
font-size:120%;}

#ygrps-yiv-89630860 #ygrps-yiv-89630860yiv3454079676 #ygrps-yiv-89630860yiv3454079676ygrp-vital ul li:last-child {
border-right:none !important;}
#ygrps-yiv-89630860



I don't know about the database, but I've seen problems with copying a dashboard output to Excel. A row's worth of data can get shoved over a column.

It would be hard to enforce, but I like to paste anything like that to Notepad, and then select all and copy to strip formatting.

You still get quotes and special characters, though.

Joe
For the most part they won't hurt anything. If you have tab characters they may affect some reports but for the most part if you are UTF you should be fine


Jose C Gomez
Software Engineer


T: 904.469.1524 mobile

Quis custodiet ipsos custodes?

On Tue, Feb 16, 2016 at 11:49 AM, ckrusen1@... [vantage] <vantage@yahoogroups.com> wrote:

Â
<div>
  
  
  <p></p><p><span>Our sales folks will often copy and past from MS Word directly the Order Entry screen, and many &quot;non-standard&quot; characters end up in the LineDescr field.</span></p><p><span>Characters like:  ™ ® © “ ” (</span><span style="word-spacing:normal;">fancy quotes), etc ...</span></p><p><span style="word-spacing:normal;"><br></span></p><p><span style="word-spacing:normal;">And occasionally  control characters like  the tab character ( &#92;t, &#92;x07, chr(7), etc ...)</span></p><p><span><br></span></p><p><span>Other then possible display issues (we&#39;ve already had the ™ appear as    â„¢   ), can these characters cause DB issues?</span></p><p><span><br></span></p><p><span>FWIW, our DB (OE 10.1b) is UTF-8.</span></p><p><span><br></span></p><p><span>Calvin</span></p><p><span> </span></p><p></p>

</div>
 


<div style="color:#fff;min-height:0;"></div>