Expand ARM Architecture Compatibility #5954
Comments
|
I am totally in favor of extending ARM architecture compatibility. This will allow Java-Tron to run on more platforms and take advantage of the benefits of the ARM architecture, such as higher energy efficiency and lower cost.
|
|
It sounds great, but I am a novice in ARM architecture. I am curious about the challenges of supporting ARM architecture. Can you list something like a task list in the future? It is convenient to clearly understand the current status and future challenges. |
|
Here are some common considerations: Important
Important 2. Native code
Tip 3. Endianness Tip 4. Memory alignment: Tip 5. Atomic operations and concurrency Caution 6. Floating-point arithmetic Tip 7. Performance optimization Important 8. Third-party dependencies
Important 9. Build and deployment process:
Tip 10. Hardware feature dependencies: Tip 11. System calls and OS interactions Important 12. Cross-platform testing
|
|
@halibobo1205 Do you want to support ARM Architecture and latest JVM version at the same time ? Or just support ARM Architecture using JDK8 ? |
|
@317787106 JVM officially supports ARM:
According to Oracle Java SE Support Roadmap, JDK9 and JDK16 are non-LTS, and JDK 17 is LTS. Based on the above information, I propose that ARM support JDK17 as a minimum. Warning This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee. |
Regarding JDK version compatibility. |
|
When is the expected completion time for this work? It sounds like a big project. |
|
@endiaoekoe I propose that ARM support JDK17 as a minimum. |
|
@abn2357 Tron currently only supports JDK 8, based on the above information, JDK17 supports ARM fully, perhaps Tron needs to upgrade JDK17 first, which is another big project. |
|
@halibobo1205 It sounds great, and I look forward to your implementation. I see that there may be subtle differences in floating point precision and behavior between ARM and x86. When supporting it, be sure to ensure data consistency. Also investigate whether there are other places that may cause data inconsistency. |
|
This issue has been added to the core devs community call #22, welcome to share the latest progress @halibobo1205, and discuss together with @endiaoekoe @tomatoishealthy @317787106 @zeusoo001 @abn2357. |
|
1. JDK version compatibility
|
|
6. Floating-point arithmetic known issues:
Unfortunately, Tron does use private long exchangeToSupply(long balance, long quant) {
logger.debug("balance: " + balance);
long newBalance = balance + quant;
logger.debug("balance + quant: " + newBalance);
double issuedSupply = -supply * (1.0 - Math.pow(1.0 + (double) quant / newBalance, 0.0005));
logger.debug("issuedSupply: " + issuedSupply);
long out = (long) issuedSupply;
supply += out;
return out;
}
private long exchangeFromSupply(long balance, long supplyQuant) {
supply -= supplyQuant;
double exchangeBalance =
balance * (Math.pow(1.0 + (double) supplyQuant / supply, 2000.0) - 1.0);
logger.debug("exchangeBalance: " + exchangeBalance);
return (long) exchangeBalance;
}Test case @Test
public void testPow() {
double x = 29218;
double q = 4761432;
double ret = Math.pow(1.0 + x / q, 0.0005);
double ret2 = StrictMath.pow(1.0 + x / q, 0.0005);
System.out.printf("%s%n", doubleToHex(ret)); // 3ff000033518c576
System.out.printf("%s%n", doubleToHex(ret2)); // 3ff000033518c575
Assert.assertEquals(0, Double.compare(ret, ret2)); // fail in jdk8_X86, success in jdk8_ARM64
}
public static String doubleToHex(double input) {
// Convert the starting value to the equivalent value in a long
long doubleAsLong = Double.doubleToRawLongBits(input);
// and then convert the long to a hex string
return Long.toHexString(doubleAsLong);
}Tron Should Use StrictMath to Avoid Cross-Platform Consistency Issues. To help ensure the portability on ARM for Java-Tron, I suggest a new proposal to convert Math to StrictMath. cc @zeusoo001 |
|
@halibobo1205 First support JDK8 on mac ARM and then extend to support JDK17 on linux and mac ARM may be smooth. |
Does this mean that there is no longer a dependency between ARM architecture upgrade and JDK upgrade? In addition, TRON only focuses on Oracle JDK, right? |
|
2. Native code JNI (Java Native Interface) or other native code.
|
|
8. Third-party dependencies Ensure all third-party libraries and dependencies support ARM architecture.
|
|
Warning This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee. To avoid subsequent charges for commercial use, I recommend switching to OpenJDK. |
|
Caution Strong data consistency and finality |
|
1. JDK version compatibility |
|
A hard fork solution will be introduced in 4.8.0, switching floating-point calculations from Math to StrictMath. |
|
Currently, java-tron supports both LevelDB and RocksDB. On the ARM architecture, we intend to support only RocksDB, mainly due to the following considerations:
This will ensure the best database usage experience on ARM architecture. |
Mac spends much time on CI test due to this: |
|
Important On the ARM architecture, we intend to support only RocksDB When using RocksDB on the CI test, we found that some tests failed due to differences in behavior between LevelDB and RocksDB.
|
Principle of X87 Instruction Simulation
Simulation implementation(c++)
Float80 ln_coe[] = {
Float80(0x3FFF, 0x8000000000000000), // 1.0
Float80(0x3FFD, 0xFFFFFFFFFFFFFFFF), // 0.5
Float80(0x3FFD, 0xAAAAAAAAAAAAAAAA), // 0.333...
Float80(0x3FFC, 0xFFFFFFFFFFFFFFFF), // 0.25
Float80(0x3FFC, 0xCCCCCCCCCCCCCCCC), // 0.2
Float80(0x3FFC, 0xAAAAAAAAAAAAAAAA), // 0.166...
};
Float80 taylor_ln2_float80(Float80 x) {
return x -
x * x * ln_coe[1] +
x * x * x * ln_coe[2] -
x * x * x * x * ln_coe[3] +
x * x * x * x * x * ln_coe[4] -
x * x * x * x * x * x * ln_coe[5];
}
// ln(2)^n/n!
Float80 exp_coe[] = {
Float80(0.69314718055994530942),
Float80(0.24022650695910071233),
Float80(0.05550410866482157995),
Float80(0.00961812910762847716),
Float80(0.00133335581464284434),
};
Float80 taylor_exp2_float80(Float80 x) {
return x * exp_coe[0] +
x * x * exp_coe[1] +
x * x * x * exp_coe[2] +
x * x * x * x * exp_coe[3] +
x * x * x * x * x * exp_coe[4];
}
double taylor_pow2_float80(double x, double y) {
Float80 x80(x), y80(y);
Float80 ln2(0x3FFE, 0xB17217F7D1CF7BBB);
Float80 y_lg2_x = y80 * taylor_ln2_float80(x80 - Float80(1)) / ln2;
// For the test dataset, since y_lg2_x<1, this step omits the exponentiation of the integer part
Float80 exp_y_lg2_x = taylor_exp2_float80(y_lg2_x) + Float80(1);
return exp_y_lg2_x.to_double();
}
Data("3ff0192278704be3", 0.0005, "3ff000033518c576"); // 4137160
Data("3ff000002fc6a33f", 0.0005, "3ff0000000061d86"); // 4065476
Data("3ff00314b1e73ecf", 0.0005, "3ff0000064ea3ef8"); // 4071538
Data("3ff0068cd52978ae", 0.0005, "3ff00000d676966c"); // 4109544
Data("3ff0032fda05447d", 0.0005, "3ff0000068636fe0"); // 4123826
Data("3ff00051c09cc796", 0.0005, "3ff000000a76c20e"); // 4166806
Data("3ff00bef8115b65d", 0.0005, "3ff0000186893de0"); // 4225778
Data("3ff009b0b2616930", 0.0005, "3ff000013d27849e"); // 4251796
Data("3ff00364ba163146", 0.0005, "3ff000006f26a9dc"); // 4257157
Data("3ff019be4095d6ae", 0.0005, "3ff0000348e9f02a"); // 4260583
Data("3ff0123e52985644", 0.0005, "3ff0000254797fd0"); // 4367125
Data("3ff0126d052860e2", 0.0005, "3ff000025a6cde26"); // 4402197
Data("3ff0001632cccf1b", 0.0005, "3ff0000002d76406"); // 4405788
Data("3ff0000965922b01", 0.0005, "3ff000000133e966"); // 4490332
Data("3ff00005c7692d61", 0.0005, "3ff0000000bd5d34"); // 4499056
Data("3ff015cba20ec276", 0.0005, "3ff00002c84cef0e"); // 4518035
Data("3ff00002f453d343", 0.0005, "3ff000000060cf4e"); // 4533215
Data("3ff006ea73f88946", 0.0005, "3ff00000e26d4ea2"); // 4647814
Data("3ff00a3632db72be", 0.0005, "3ff000014e3382a6"); // 4766695
Data("3ff000c0e8df0274", 0.0005, "3ff0000018b0aeb2"); // 4771494
Data("3ff00015c8f06afe", 0.0005, "3ff0000002c9d73e"); // 4793587
Data("3ff00068def18101", 0.0005, "3ff000000d6c3cac"); // 4801947
Data("3ff01349f3ac164b", 0.0005, "3ff000027693328a"); // 4916843
Data("3ff00e86a7859088", 0.0005, "3ff00001db256a52"); // 4924111
Data("3ff00000c2a51ab7", 0.0005, "3ff000000018ea20"); // 5098864
Data("3ff020fb74e9f170", 0.0005, "3ff00004346fbfa2"); // 5133963
Data("3ff00001ce277ce7", 0.0005, "3ff00000003b27dc"); // 5139389
Data("3ff005468a327822", 0.0005, "3ff00000acc20750"); // 5151258
Data("3ff00006666f30ff", 0.0005, "3ff0000000d1b80e"); // 5185021
Data("3ff000045a0b2035", 0.0005, "3ff00000008e98e6"); // 5295829
Data("3ff00e00380e10d7", 0.0005, "3ff00001c9ff83c8"); // 5380897
Data("3ff00c15de2b0d5e", 0.0005, "3ff000018b6eaab6"); // 5400886
Data("3ff00042afe6956a", 0.0005, "3ff0000008892244"); // 5864127
Data("3ff0005b7357c2d4", 0.0005, "3ff000000bb48572"); // 6167339
Data("3ff00033d5ab51c8", 0.0005, "3ff0000006a279c8"); // 6240974
Data("3ff0000046d74585", 0.0005, "3ff0000000091150"); // 6279093
Data("3ff0010403f34767", 0.0005, "3ff0000021472146"); // 6428736
Data("3ff00496fe59bc98", 0.0005, "3ff000009650a4ca"); // 6432355,6493373
Data("3ff0012e43815868", 0.0005, "3ff0000026af266e"); // 6555029
Data("3ff00021f6080e3c", 0.0005, "3ff000000458d16a"); // 7092933
Data("3ff000489c0f28bd", 0.0005, "3ff00000094b3072"); // 7112412
Data("3ff00009d3df2e9c", 0.0005, "3ff00000014207b4"); // 7675535
Data("3ff000def05fa9c8", 0.0005, "3ff000001c887cdc"); // 7860324
Data("3ff0013bca543227", 0.0005, "3ff00000286a42d2"); // 8292427
Data("3ff0021a2f14a0ee", 0.0005, "3ff0000044deb040"); // 8517311
Data("3ff0002cc166be3c", 0.0005, "3ff0000005ba841e"); // 8763101
Data("3ff0000cc84e613f", 0.0005, "3ff0000001a2da46"); // 9269124
Data("3ff000057b83c83f", 0.0005, "3ff0000000b3a640"); // 9631452
exp:0.0005 base:1.00628495434413 3ff019be4095d6ae
expected: 1.0000031326481 3ff0000348e9f02a
result: 1.0000031326481 3ff0000348e9f029
exp:0.0005 base:1.00805230779141 3ff020fb74e9f170
expected: 1.00000401003852 3ff00004346fbfa2
result: 1.00000401003852 3ff00004346fbfa1
|
|
diff data(48 POW calculation instances) is the result of Math and StrictMath, and if algorithmic simulations are performed, the implementation needs to be fully tested for full equivalence with the Math library. |
|
|
In the Java HotSpot virtual machine, Intrinsics in the HotSpot JVM are special, optimized implementations of commonly used Java methods. When the JVM identifies specific method calls, it may replace the standard Java implementation of these methods with more efficient native code. Specifically for the
This optimization mechanism is usually managed by the do_intrinsic(_dlog, java_lang_Math, log_name, double_double_signature, F_S)Through this intrinsic function optimization, the HotSpot JVM allows Java programs to maintain platform independence while achieving performance close to native code, which is particularly beneficial for math-computation-intensive applications. Here's the logic for |
|
@halibobo1205 When Math calculates pow(double,double), how can you determine if the result is inconsistent with that calculated by StrictMath? What to do when inconsistencies are found? And you can specify what's hardcoded. |
public class ExchangeCapsule implements ProtoCapsule<Exchange> {
public long transaction(byte[] sellTokenID, long sellTokenQuant, boolean useStrictMath) {
long supply = 1_000_000_000_000_000_000L;
ExchangeProcessor processor = new ExchangeProcessor(supply, useStrictMath);
ExchangeProcessor strictProcessor = new ExchangeProcessor(supply, true);
long buyTokenQuant = 0;
long strictBuyTokenQuant = 0;
long firstTokenBalance = this.exchange.getFirstTokenBalance();
long secondTokenBalance = this.exchange.getSecondTokenBalance();
if (this.exchange.getFirstTokenId().equals(ByteString.copyFrom(sellTokenID))) {
buyTokenQuant = processor.exchange(firstTokenBalance,
secondTokenBalance,
sellTokenQuant);
strictBuyTokenQuant = strictProcessor.exchange(firstTokenBalance,
secondTokenBalance,
sellTokenQuant);
if (!useStrictMath && buyTokenQuant != strictBuyTokenQuant) {
logAndRecord("{}\t{}\t{}\t{}\t{}", buyTokenQuant, strictBuyTokenQuant, firstTokenBalance, secondTokenBalance, sellTokenQuant); // logAndRecord pow data
}
this.exchange = this.exchange.toBuilder()
.setFirstTokenBalance(firstTokenBalance + sellTokenQuant)
.setSecondTokenBalance(secondTokenBalance - buyTokenQuant)
.build();
} else {
buyTokenQuant = processor.exchange(secondTokenBalance,
firstTokenBalance,
sellTokenQuant);
strictBuyTokenQuant = strictProcessor.exchange(secondTokenBalance,
firstTokenBalance,
sellTokenQuant);
if (!useStrictMath && buyTokenQuant != strictBuyTokenQuant) {
logAndRecord("{}\t{}\t{}\t{}\t{}", buyTokenQuant, strictBuyTokenQuant,secondTokenBalance, firstTokenBalance, sellTokenQuant); // logAndRecord pow data
}
this.exchange = this.exchange.toBuilder()
.setFirstTokenBalance(firstTokenBalance - buyTokenQuant)
.setSecondTokenBalance(secondTokenBalance + sellTokenQuant)
.build();
}
return buyTokenQuant;
}
public class ExchangeProcessor {
private long supply;
private final boolean useStrictMath;
public ExchangeProcessor(long supply, boolean useStrictMath) {
this.supply = supply;
this.useStrictMath = useStrictMath;
}
private long exchangeToSupply(long balance, long quant) {
long newBalance = balance + quant;
double issuedSupply = -supply * (1.0 - Maths.pow(1.0 + (double) quant / newBalance, 0.0005, this.useStrictMath));
long out = (long) issuedSupply;
supply += out;
return out;
}
private long exchangeFromSupply(long balance, long supplyQuant) {
supply -= supplyQuant;
double exchangeBalance = balance * (Maths.pow(1.0 + (double) supplyQuant / supply, 2000.0, this.useStrictMath) - 1.0);
return (long) exchangeBalance;
}
public long exchange(long sellTokenBalance, long buyTokenBalance, long sellTokenQuant) {
long relay = exchangeToSupply(sellTokenBalance, sellTokenQuant);
return exchangeFromSupply(buyTokenBalance, relay);
}
}
private static final Map<Double, Double> powData = Collections.synchronizedMap(new HashMap<>());
public static double pow(double a, double b) {
double strictResult = StrictMath.pow(a, b);
return powData.getOrDefault(a, strictResult);
}
} |
|
If there are other ways to implement X87 Instruction Simulation, please discuss them. |
|
@halibobo1205 I noticed that the pow results in |
Yes, precision loss precisely reduces the amount of the pow data that needs to be hard-coded. |
Principle of X87 Instruction SimulationUSE MPFR mpfr_pow#include <stdio.h>
#include <gmp.h>
#include <mpfr.h>
// Precision settings
#define X87_PRECISION 64 // 64-bit mantissa for x87 80-bit format
int main(void) {
mpfr_t base1, exp1, result1;
mpfr_t base2, exp2, result2;
mpfr_set_default_prec(X87_PRECISION);
mpfr_set_default_rounding_mode(MPFR_RNDN);
mpfr_init2(base1, X87_PRECISION);
mpfr_init2(exp1, X87_PRECISION);
mpfr_init2(result1, X87_PRECISION);
mpfr_init2(base2, X87_PRECISION);
mpfr_init2(exp2, X87_PRECISION);
mpfr_init2(result2, X87_PRECISION);
mpfr_set_d(base1, 1.0061363892207218, MPFR_RNDN);
mpfr_set_d(exp1, 0.0005, MPFR_RNDN);
mpfr_set_d(base2, 1.0000046943914231, MPFR_RNDN);
mpfr_set_d(exp2, 2000, MPFR_RNDN);
mpfr_pow(result1, base1, exp1, MPFR_RNDN);
mpfr_pow(result2, base2, exp2, MPFR_RNDN);
printf("pow(1.0061363892207218, 0.0005) = ");
mpfr_out_str(stdout, 10, 17, result1, MPFR_RNDN);
printf("\n");
printf("pow(1.0000046943914231, 2000) = ");
mpfr_out_str(stdout, 10, 17, result2, MPFR_RNDN);
printf("\n");
mpfr_clear(base1);
mpfr_clear(exp1);
mpfr_clear(result1);
mpfr_clear(base2);
mpfr_clear(exp2);
mpfr_clear(result2);
mpfr_free_cache();
return 0;
}❌ Unable to precisely simulate x86 pow |
Principle of X87 Instruction SimulationUSE Apfloat❌ The short answer is "no", see details: 
|
cc @NewOF |
|
According to the documentation of the 8087 support library mentioned in the link, it provides a simulated implementation of the relevant instructions. However, since the corresponding source code is not provided, it is not possible to verify the correctness. |
|
Hard-Code: |
|
Some constants mentioned in another link do not provide precise hexadecimal representations; instead, they use decimal floating-point numbers with insufficient precision, which are not very meaningful for our calculation scenario. |
|
Floating-point arithmetic Important For historical data(the Bancor trading pair), Hardcoded Special Cases(48 POW calculation instances) Hard-coding remains the optimal solution at this stage, though we'll continue exploring alternative approaches. I welcome your thoughts and input on this matter. |
|
All current progress is documented in PR #6327. We welcome any new ideas or suggestions for further improvement. |
|
Milestone Update (2025-05-16)
|
|
Milestone Update (2025-05-23)
|
@halibobo1205 Hi, What's the current progress on this test? Looking forward to seeing the test results for this part, as this outcome is quite important. |
Cross-platform testing
|
Great! Do we have a detailed schedule for the remaining work on this feature before it’s released? And is there any way the community can get involved in reviewing or testing? |
OK, thanks for the information! Maybe we can add more tests to cover more scenarios. |
Uh oh!
There was an error while loading. Please reload this page.
Background
Java-Tron currently only supports the x86 architecture. Nevertheless, ARM architecture has gained significant traction recently, especially in cloud computing and mobile devices. ARM processors are known for their energy efficiency and cost-effectiveness, making them increasingly popular in data centers, cloud computing, and edge computing scenarios. It will be great to have an option to run Java-Tron using the ARM architecture.
Key developments in ARM architecture:
ARM advantages:
Related Issues and PRs
Scope of Impact
Current Progress Summary
JDK version
Native code
Third-party dependencies
Floating-point arithmetic
Build and deployment process
The text was updated successfully, but these errors were encountered: