Handling small decimal numbers

If you think that something doesn't work in Aware IM post your message here
Post Reply
JoshK131
Posts: 31
Joined: Tue Jan 26, 2021 11:09 pm

Handling small decimal numbers

Post by JoshK131 »

Hello,

I am trying to accurately store very small decimal numbers but the numbers are automatically changed to "#.#" format.

i.e.
0.000000123 becomes 1.23
Decimal needs at least 6 0's
The database stores the modified value (1.23)

Below is a gif of it happening.
https://puu.sh/JlwVy/2d8d01d8cd.gif

To Replicate the problem:
1) Create a number field in an attribute
2) format is #.############### (other formats may have the same problem)
3) Make sure number field is in the form
4) run test version and open form
5) input small decimal with 6 or more 0's after the decimal point
6) Save form

Is there a way for me to handle this behavior?

Another problem is aware im seems to handle numbers with a precision of 16. Is there a way to increase that?
chris__29
Posts: 20
Joined: Wed Dec 15, 2021 11:44 pm
Location: Australia

Re: Handling small decimal numbers

Post by chris__29 »

try

#0.000000
AwareIM 8.8
MySQL, MSSQL
MS Server
Australia
JoshK131
Posts: 31
Joined: Tue Jan 26, 2021 11:09 pm

Re: Handling small decimal numbers

Post by JoshK131 »

Setting the format to 0.00... does not help either.

Thanks for the suggestion.
ACDC
Posts: 1138
Joined: Sat Jun 30, 2007 5:03 pm
Location: California, USA

Re: Handling small decimal numbers

Post by ACDC »

If you set your number attribute to "decimal" you will have more control over the rounding aspect stored in the DB :

e.g. on the number setup, under Advanced, change SQL Type to: decimal (19,4) or configure the decimal type accordingly
JoshK131
Posts: 31
Joined: Tue Jan 26, 2021 11:09 pm

Re: Handling small decimal numbers

Post by JoshK131 »

The SQL type for the number is currently set to "DECIMAL (40,20)"
That does not work.

"Real" type does not work either.
ACDC
Posts: 1138
Joined: Sat Jun 30, 2007 5:03 pm
Location: California, USA

Re: Handling small decimal numbers

Post by ACDC »

What does it store in the database ?

I work with 4 digits after the decimal and use custom setup of the number format to: #,##0.0000
JoshK131
Posts: 31
Joined: Tue Jan 26, 2021 11:09 pm

Re: Handling small decimal numbers

Post by JoshK131 »

The database also stores 1.234

The bug happens with 6 digits after the decimal.
Post Reply