vefatica May 12,934 170 Aug 16, 2017 #1 How many decimal places doe @EVAL use internally? I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places?
How many decimal places doe @EVAL use internally? I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places?
rconn Administrator May 13,063 194 Staff member Aug 16, 2017 #2 vefatica said: How many decimal places doe @EVAL use internally? I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places? Click to expand... None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers).
vefatica said: How many decimal places doe @EVAL use internally? I mean, if I use @EVAL[w+x+y+z=2], does it add four precision-2 numbers or does it use higher precision and round the result to 2 places? Click to expand... None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers).
vefatica May 12,934 170 Aug 16, 2017 #3 rconn said: None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers). Click to expand... I sorta knew that, but don't BCDs have some precision limit? And, if so, is it always the same?
rconn said: None of the above. @EVAL uses BCD (except for a handful of integer-only functions that use 64-bit integers). Click to expand... I sorta knew that, but don't BCDs have some precision limit? And, if so, is it always the same?
rconn Administrator May 13,063 194 Staff member Aug 16, 2017 #4 The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL.
The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL.
vefatica May 12,934 170 Aug 16, 2017 #5 I should also have asked ... it is correct, then, that a precision spec, like "=5" affects only the output, and not the calculation itself?
I should also have asked ... it is correct, then, that a precision spec, like "=5" affects only the output, and not the calculation itself?
vefatica May 12,934 170 Aug 16, 2017 #6 rconn said: The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL. Click to expand... Same question about that ... does it affect only the output and not the calculation itself?
rconn said: The precision limit for @EVAL is 10,000 decimal places. You set the precision to use in OPTION or in @EVAL. Click to expand... Same question about that ... does it affect only the output and not the calculation itself?
vefatica May 12,934 170 Aug 16, 2017 #7 Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little, 1. ExpandVariables(L"%@EVAL[expression]", 1) 2. ExpandVariables(expression, 1); Evaluate(expression);
Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little, 1. ExpandVariables(L"%@EVAL[expression]", 1) 2. ExpandVariables(expression, 1); Evaluate(expression);
rconn Administrator May 13,063 194 Staff member Aug 16, 2017 #8 vefatica said: Same question about that ... does it affect only the output and not the calculation itself? Click to expand... Precision is only for output. Internally @EVAL uses whatever precision is necessary for the calculation.
vefatica said: Same question about that ... does it affect only the output and not the calculation itself? Click to expand... Precision is only for output. Internally @EVAL uses whatever precision is necessary for the calculation.
rconn Administrator May 13,063 194 Staff member Aug 16, 2017 #9 vefatica said: Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little, 1. ExpandVariables(L"%@EVAL[expression]", 1) 2. ExpandVariables(expression, 1); Evaluate(expression); Click to expand... Yes.
vefatica said: Another, mostly unrelated question about @EVAL ... in a plugin, are these equivalent. Paraphrasing a little, 1. ExpandVariables(L"%@EVAL[expression]", 1) 2. ExpandVariables(expression, 1); Evaluate(expression); Click to expand... Yes.