I've been struggling with this for some time now (years, that is): The precision of calculations performed in D3.
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
Page 1 / 1
I've been struggling with this for some time now (years, that is): The precision of calculations performed in D3.
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
Previously when needing e.g. precision of 8, then I would
1. multiply the dividend by 10000,
use the result as a new dividend
2. perform the division using the same divisor but on the new dividend
3. perform my query (If, etc) on the new quotient
The original dividend is untouched.
for example:
21.00 / 5.1 = 4.1176
however,
210000.00/5.1 = 41,176.4706
Hope this helps
Merry
On Tue, 29 Jun 2021 at 01:01, Walter Kiess via Rocket Forum <
Mail@forum.rocketsoftware.com> wrote:
> I've been struggling with this for some time now (years, that is): The
> precision of calculations performed in D3. As everyone knows, D3 performs...
> Be sure to join the forums you're interested in to be notified of new
> content. Click the join button from either the forum listing page or the
> home page of any given sub-forum.
>
> Tip: Want a single update on all your forum memberships? Go to Profile >
> My Account > Forum Notifications, and check 'daily consolidated digest.'
> Switch the discussion email drop down to 'no email' or you will receive
> both.
> ------------------------------
> [image: Rocket Software]
> D3 and mvBase
>
> Post New Message Online
>
> Precision in Numerical Calculations in D3
>
> Reply to Group Online
>
> to Group
>
> [image: Walter Kiess]
>
> Jun 28, 2021 8:02 PM
> Walter Kiess
>
> I've been struggling with this for some time now (years, that is): The
> precision of calculations performed in D3.
>
> As everyone knows, D3 performs calculations to a default of 4 decimal
> places. It then truncates the result so everything past the 4th decimal
> place is removed. This in itself is a cause of concern as it introduces the
> 1st of several calculation inaccuracies. From my understanding, this occurs
> for every calculation performed, whether it's in a single expression or in
> separate statements. Thus there may be several inaccuracies in a single
> expression, particularly when fractions or division is involved.
>
> To get around this, I have arranged my expressions such that division is
> always performed last (where possible). But this doesn't solve the problem
> completely though. There is always an intrinsic error in the
> calculation/expression due to the truncation as the expression is not
> calculated as a floating-point expression. When you're dealing with
> hundreds of thousands or even millions of dollars, this becomes a problem,
> as even a small error will have a large effect on the outcome of the
> calculation.
>
> D3 does have the ability to change the precision of its calculations, but
> there is a caveat: ALL programs in the system must use the same PRECISION.
> You can't just call an external subroutine with higher precision to perform
> the calls and return a properly-rounded value. That would cause an error
> due to the different precision of the calling and called programs.
>
> What are developers doing to ensure a higher precision for their
> calculations? What's your "secret sauce"?
>
> ------------------------------
> Walter Kiess
> Manager IT
> SA Police Super
> Adelaide SA Australia
> ------------------------------
> *Reply to Group Online
>
> *View Thread
>
> *Recommend
>
> *Forward
>
> *Flag as Inappropriate
>
> *Post New Message Online
>
>
>
>
>
> You are subscribed to "D3 and mvBase" as merry.player@gmail.com. To
> change your subscriptions, go to My Subscriptions
>
> To unsubscribe from this community discussion, go to Unsubscribe
>
> .
>
--
+44 (0)7774212819
07774212819
https://www.linkedin.com/in/merryplayer
https://www.merryplayer.com
I've been struggling with this for some time now (years, that is): The precision of calculations performed in D3.
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
- Use a higher precisone in all your programs: I use PRECISION 6.
- Compile all programs in FlashBASIC.
01 PROGRAM PP1 02 PRECISION 4 03 A=1 04 CALL SS1(A) 05 CRT "A=":A ;* store it in DB, don't use it as a number or it will be 0.3333 06 STOP 01 SUB SS1(A) 02 PRECISION 9 03 A/=3 ;* 0.333333333 04 A=A:"" ;* make it string 05 RETURN - Multiply at least one factor by 1 billion or more (to force D3 to use string math), do all calulations, "scale" to your precison (maybe this operations could be a "string manipulation")
- Use "String Arithmetic functions" (AccuMath) if You need precision higher than 9
------------------------------
Stefano Maran
GTN Spa
------------------------------
Four suggestions...
------------------------------
Stefano Maran
GTN Spa
------------------------------
- Use a higher precisone in all your programs: I use PRECISION 6.
- Compile all programs in FlashBASIC.
01 PROGRAM PP1 02 PRECISION 4 03 A=1 04 CALL SS1(A) 05 CRT "A=":A ;* store it in DB, don't use it as a number or it will be 0.3333 06 STOP 01 SUB SS1(A) 02 PRECISION 9 03 A/=3 ;* 0.333333333 04 A=A:"" ;* make it string 05 RETURN - Multiply at least one factor by 1 billion or more (to force D3 to use string math), do all calulations, "scale" to your precison (maybe this operations could be a "string manipulation")
- Use "String Arithmetic functions" (AccuMath) if You need precision higher than 9
------------------------------
Stefano Maran
GTN Spa
------------------------------
When i need to use multple precisions i use the ICONV for it
passing like this RESULT = ICONV(VARS,"NEW PROGRAM")
like this it dosent matter the PRECISION of the calling program and the subprogram
------------------------------
Alberto Leal
Systems Analyst
Millano distribuidora
Campo Grande Brazil
------------------------------
I've been struggling with this for some time now (years, that is): The precision of calculations performed in D3.
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
Here's what I did when faced with this way back around 1990, when working with Pick R83 to create an FMIS / ERP I called "Crusader". I was faced with having to deal with precision of between 0 and 6 decimals specified by data dictionary, and able to be changed without changing the code. The following approach has survived the years since then (still being in use today) ...
1. I save all numeric values as whole numbers (by that I mean with no decimals stored) by using the dictionary specification then ICONV / OCONV to handle the necessary conversions for display / print. I guess this has relatively little to do with solving the identified problem, but it saves database space (which was an issue in 1990) and simplifies routines if they only ever deal with whole numbers. So that leads on to ...
2. If I could see that I faced a potential loss of precision in specific situations, I multiplied one or more factors first (by perhaps 1000), then in my code, did appropriate rounding (up / down) in the final result just before I applied a correction for whatever I had multiplied by to start with.
What forced this approach was that all my routines assume nothing about the origin or nature of the data they handle. My product was totally driven by the data dictionaries, which I extended to include a lot more functionality.
To support the flexibility I wanted, I developed a routine to be able to universally handle correlatives, including those that OCONV cannot. I have attached my source code for this routine. It includes code that is specific to my product, but if you analyse what I do in that case, you will see how to link other code that extends a data dictionary.
I hope that helps some?
------------------------------
Alan Miles
Business and IT Consultant
Amba Limited
Whanganui, New Zealand
------------------------------
Four suggestions...
------------------------------
Stefano Maran
GTN Spa
------------------------------
- Use a higher precisone in all your programs: I use PRECISION 6.
- Compile all programs in FlashBASIC.
01 PROGRAM PP1 02 PRECISION 4 03 A=1 04 CALL SS1(A) 05 CRT "A=":A ;* store it in DB, don't use it as a number or it will be 0.3333 06 STOP 01 SUB SS1(A) 02 PRECISION 9 03 A/=3 ;* 0.333333333 04 A=A:"" ;* make it string 05 RETURN - Multiply at least one factor by 1 billion or more (to force D3 to use string math), do all calulations, "scale" to your precison (maybe this operations could be a "string manipulation")
- Use "String Arithmetic functions" (AccuMath) if You need precision higher than 9
------------------------------
Stefano Maran
GTN Spa
------------------------------
I can confirm that Stefano's code example works using FlashBASIC, however, setting a different PRECISION value in subroutines using Standard BASIC Object code will abort with the runtime error:
[B33] in program "PP1", Line 5: Precision declared in subprogram 'SS1'
is different from that declared in the mainline program.
Cheers,
-Mike
------------------------------
Mike Raffaele
Meier Business Systems
------------------------------
I've been struggling with this for some time now (years, that is): The precision of calculations performed in D3.
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
As everyone knows, D3 performs calculations to a default of 4 decimal places. It then truncates the result so everything past the 4th decimal place is removed. This in itself is a cause of concern as it introduces the 1st of several calculation inaccuracies. From my understanding, this occurs for every calculation performed, whether it's in a single expression or in separate statements. Thus there may be several inaccuracies in a single expression, particularly when fractions or division is involved.
To get around this, I have arranged my expressions such that division is always performed last (where possible). But this doesn't solve the problem completely though. There is always an intrinsic error in the calculation/expression due to the truncation as the expression is not calculated as a floating-point expression. When you're dealing with hundreds of thousands or even millions of dollars, this becomes a problem, as even a small error will have a large effect on the outcome of the calculation.
D3 does have the ability to change the precision of its calculations, but there is a caveat: ALL programs in the system must use the same PRECISION. You can't just call an external subroutine with higher precision to perform the calls and return a properly-rounded value. That would cause an error due to the different precision of the calling and called programs.
What are developers doing to ensure a higher precision for their calculations? What's your "secret sauce"?
------------------------------
Walter Kiess
Manager IT
SA Police Super
Adelaide SA Australia
------------------------------
https://www3.rocketsoftware.com/rocketd3/support/documentation/d3nt/103/refman/d3/d3_all/source/reference_manual/accumath/Accumath.htm
Have a look in the 'Basic Subroutine' section
https://www3.rocketsoftware.com/rocketd3/support/documentation/d3nt/103/refman/d3/d3_all/source/reference_manual/accumath/BASIC_subroutines.htm
Personally, I've always found prescaling numbers to be counter-intuitive and confusing later on.
Unfortunately, you will need to deconstruct the maths into operator and arguments, to be able to call the subroutines.
Would be nice if there was some form arithmetic parser that could handle variables too.
I did try my hand at that, but didn't have the luxury to spend time.
Rgrds M
------------------------------
mario bajada
abc tissue products
------------------------------
Sign up
Already have an account? Login
Welcome to the Rocket Forum!
Please log in or register:
Employee Login | Registration Member Login | RegistrationEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.
77 4th Avenue
Waltham, MA 02451 USA
