Problem: Quey Unicode data

docthusinh

Dear all,

My program need to store and retrieve data in Unicode encoding. I used SQL server with collation Vietnamese_CI_AS, and for add record as well as query all record, Aware IM work perfectly.

But when it come to query a specific word in unicode format, the system could not find it, any suggestion?

aware_support

Can you use the SQL Server tools to check what SQL query is being run and attach this query?

docthusinh

I've also tried to modify UTF-8 encoding value in:

standardStartHead.jsp
C:\AwareIM\Tomcat\webapps\AwareIM\WEB-INF\web.xml
C:\AwareIM\Tomcat\conf\server.xml

docthusinh

Got XML request <root><set_prompts_action>
<query_string>FIND Employee WHERE Employee.Name CONTAINS ?Name</query_string>
<prompted_value name="Name" >Kh??ng</prompted_value>
</set_prompts_action>
</root>
WebServlet URL got request with params:
Exception thrown com.bas.shared.ruleparser.ParseException Lexical error at line 1, column 47. Encountered: "\u01b0" (432), after : "\'Kh"
com.bas.shared.ruleparser.ParseException: Lexical error at line 1, column 47. Encountered: "\u01b0" (432), after : "\'Kh"
at com.bas.shared.domain.configuration.elements.Query.setRuleLanguageString(Unknown Source)
at com.bas.shared.domain.configuration.elements.Query.createFromRuleLanguageString(Unknown Source)
at com.bas.webapp.thin.a.c.h.a(Unknown Source)
at com.bas.webapp.thin.a.b.a(Unknown Source)
at com.bas.webapp.thin.a.b.a(Unknown Source)
at com.bas.webapp.thin.servlets.WebInterfaceURLServlet.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188-)
at CharsetFilter.doFilter(CharsetFilter.java:23)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188-)
at CharsetFilter.doFilter(CharsetFilter.java:23)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188-)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108-)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528-)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:595)
Web server returned error: Lexical error at line 1, column 47. Encountered: "\u01b0" (432), after : "\'Kh"
Got XML request <root><query_layout_action query_name='QueryEmployee' >
</query_layout_action>
</root>

aware_support

This is not what I meant.

In Aware IM perform a search that will not display the data as expected
Go to the SQL Server profiler (I think this is the tool) and check SQL queries that were run against the BASDB or BASDBTEST databases. Check the SQL that Aware IM created for the query

I need to verify that the SQL was correct.

docthusinh

Hi,

There are no records on sql server.

I think the characters could not be pass through XML request.

String should be Khương not Kh??ng as below log from AwareIM Control Panel

Got XML request <root><set_prompts_action>
<query_string>FIND Employee WHERE Employee.Name CONTAINS ?Name ORDER BY Employee.Name</query_string>
<prompted_value name="Name" >Kh??ng</prompted_value>
</set_prompts_action>
</root>
WebServlet URL got request with params:
Exception thrown com.bas.shared.ruleparser.ParseException Lexical error at line 1, column 47. Encountered: "\u01b0" (432), after : "\'Kh"
com.bas.shared.ruleparser.ParseException: Lexical error at line 1, column 47. Encountered: "\u01b0" (432), after : "\'Kh"

aware_support

Yes, it's a bug and a difficult one. We are working on it

aware_support

Please try this in build 1246

docthusinh

Hi,
I have no luck with the version mentioned. This time it has no lexical error, but somehow those XML request could not doing well the character encoding thing.

Got XML request <root><form_search_action object_name='emp' query_name='searchemp' >

<post_attribute_data attribute_name="name" >t???ng</post_attribute_data>

</form_search_action>

</root>

Attribute name value t???ng

Form search query: FIND emp WHERE emp.name CONTAINS 't???ng'

[/quote]

aware_support

All Unicode search problems should be fixed in build 1351