Skip to content

Missing Android N compatibility because of behavior changes #216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TheNephilim88 opened this issue Mar 10, 2016 · 77 comments
Closed

Missing Android N compatibility because of behavior changes #216

TheNephilim88 opened this issue Mar 10, 2016 · 77 comments

Comments

@TheNephilim88
Copy link

Hello,

Google changed some things regarding linking/loading libs: https://developer.android.com/preview/behavior-changes.html#ndk

This also affects SQLCipher. E.g. the Test-Suite failed 10 out of 32 tests.
screenshot_20160310-091222

Log show up following:

03-10 09:09:17.497 26856-26869/net.zetetic:provider W/linker: library "libutils.so" ("/system/lib/libutils.so") needed or dlopened by "/data/app/net.zetetic-1/lib/arm/libsqlcipher_android.so" is not accessible for the namespace "classloader-namespace" - the access is temporarily granted as a workaround for http://b/26394120
03-10 09:09:17.497 26856-26869/net.zetetic:provider W/linker: library "libcutils.so" ("/system/lib/libcutils.so") needed or dlopened by "/data/app/net.zetetic-1/lib/arm/libsqlcipher_android.so" is not accessible for the namespace "classloader-namespace" - the access is temporarily granted as a workaround for http://b/26394120
03-10 09:09:17.517 26856-26869/net.zetetic:provider W/linker: library "libnativehelper.so" ("/system/lib/libnativehelper.so") needed or dlopened by "/data/app/net.zetetic-1/lib/arm/libdatabase_sqlcipher.so" is not accessible for the namespace "classloader-namespace" - the access is temporarily granted as a workaround for http://b/26394120
03-10 09:09:17.518 26856-26869/net.zetetic:provider W/linker: library "libandroid_runtime.so" ("/system/lib/libandroid_runtime.so") needed or dlopened by "/data/app/net.zetetic-1/lib/arm/libdatabase_sqlcipher.so" is not accessible for the namespace "classloader-namespace" - the access is temporarily granted as a workaround for http://b/26394120
03-10 09:09:17.518 26856-26869/net.zetetic:provider W/linker: library "libbinder.so" ("/system/lib/libbinder.so") needed or dlopened by "/data/app/net.zetetic-1/lib/arm/libdatabase_sqlcipher.so" is not accessible for the namespace "classloader-namespace" - the access is temporarily granted as a workaround for http://b/26394120
@developernotes
Copy link
Member

Hello @TheNephilim88

Thank you for your early report of SQLCipher for Android on Android N. We hope to begin looking into the Android N build soon. Take care!

@splendidbits
Copy link

As per https://developer.android.com/preview/behavior-changes.html#ndk

Use of SSL_ctrl symbol from libcrypto.so should be replaced with an app local version. For example, you should statically link libcyrpto.a in your .so file or include your own dynamically libcrypto.so from BoringSSL or OpenSSL in your app.

So instead of OpenSSL being compiled for each arch and being statically linked in the .so, I believe now that the bordingSSL library so (or a?) should be linked directly through the ndk from the user's OS. OpenSSL has been deprecated and removed in favour of boringssl

@developernotes
Copy link
Member

Hi @staticfish

We have historically built a static OpenSSL because of the differing versions of OpenSSL bundled across the various Android OS's. We will consider this change in the future.

@TheNephilim88
Copy link
Author

Did anyone found time to look into this?
I know Android N is still dev preview, but its annoying nonetheless, especially if users dont understand and are giving bad ratings because of it :(

@sjlombardo
Copy link
Member

sjlombardo commented Mar 19, 2016 via email

@neutron666
Copy link

But that's what dev previews are for, make issues appear early and fix them. I really would like to continune testing the N dev preview, but i need a app based on this lib urgently and it's very annoying to being able to use it. PLEASE start on fixing this problem as soon as possible.

@mikecousins
Copy link

Hopefully this gets prioritized soon. The ease for user's to install the latest dev preview has resulted in quite a few people not being able to run anything using your library.

@smarkovik
Copy link

Hello guys,
any ideas when this will get in your pipeline?
N preview has been out for a while and the apps which depend on sql cipher are essentially blocked.

@sjlombardo
Copy link
Member

@smarkovik we just finished up a release of SQLCipher yesterday. Now that is out of the way, we'll be switching over to look at this issue as a priority.

@smarkovik
Copy link

Hello @sjlombardo,
Not being pushy but just curious of your progress? any ETAs?
could the community help ?

@developernotes
Copy link
Member

Hi @smarkovik

We are actively working on this project and will make our changes available as soon as everything is working. Thanks!

@mvojjala
Copy link

Hi @sjlombardo

We are also in same situation. Not able use our app on Android N because of this issue. Just want to check with you if you have any update or tentative dates of a possible update.

Thanks

@sjlombardo
Copy link
Member

@mvojjala we're making appreciable progress on this issue, but it does involve some fairly major changes. Keep an eye on this ticket, and/or at https://discuss.zetetic.net/c/sqlcipher for information about upcoming availability for testing.

@fuentespatrick
Copy link

I appreciate the effort and the update @sjlombardo.
My team is similarly blocked, so the significant work is greatly appreciated!

@developernotes
Copy link
Member

Hi folks,

We have just announced a beta of SQLCipher for Android with support for Android Developer N Preview here. Please take a look if you are interested in helping test out these changes. Thanks!

@smarkovik
Copy link

Awsome work guys !

@fuentespatrick
Copy link

Thanks, we'll give it a try!

@zokipirlo
Copy link

Tnx. I think it will work, but I have also a dependency on IOCipher. It's crashing while loading stlport_shared inside static block in VirtualFileSystem.java. I tried to remove it, but it's still loading it in iocipher. Is there any easy solution to make it work?

@developernotes
Copy link
Member

Hi @zokipirlo

It sounds as if IOCipher may need to be rebuilt referencing the new version of SQLCipher for Android. IOCipher has publish build instructions on their README here.

@zokipirlo
Copy link

It works great. Everything looks fine. Thank you.
I was having some problems with building IOCipher but that's another story :) Doesn't matter.

@developernotes
Copy link
Member

Hello @zokipirlo

Thanks for the report, we are glad to hear it is working well for you!

@brody4hire
Copy link

I am also really happy that you guys made the solution. I am especially happy that you guys removed the ICU and native setLocale parts, though it may be nice to have a version with ICU for those who may need it. I encounter 3 minor failures when I ran the SQLCipher test suite:

  • Unicode (ICU) test
  • Closed database test
  • Invalid password test

The following change would the invalid password test:

--- a/src/main/java/net/zetetic/ZeteticApplication.java
+++ b/src/main/java/net/zetetic/ZeteticApplication.java
@@ -53,7 +53,10 @@ public class ZeteticApplication extends Application {
     public SQLiteDatabase createDatabase(File databaseFile){
         Log.i(TAG, "Entered ZeteticApplication::createDatabase");
         Log.i(TAG, "Before SQLiteDatabase.openOrCreateDatabase");
-        return SQLiteDatabase.openOrCreateDatabase(databaseFile, DATABASE_PASSWORD, null);
+        SQLiteDatabase database = SQLiteDatabase.openOrCreateDatabase(databaseFile, DATABASE_PASSWORD, null);
+        Log.i(TAG, "Populating the database");
+        database.rawExecSQL("CREATE TABLE IF NOT EXISTS MyTest (TestData)");
+        return database;
     }

     public void extractAssetToDatabaseDirectory(String fileName) throws IOException {

The closed database test fails because it tries the SQLiteDatabase.setLocale function which then throws an java.lang.UnsatisfiedLinkError with a "Native method not found" message. Considering the native_setLocale method is no longer implemented, I would favor fixing SQLiteDatabase.setLocale to just throw a runtime exception with a message that the method is no longer implemented.

@developernotes
Copy link
Member

Hi @brodybits

Thank you for your feedback, it is much appreciated! Have you pulled down the latest changes to the SQLCipher for Android test suite? It should address those tests and also include a few new tests too.

@brody4hire
Copy link

Have you pulled down the latest changes to the SQLCipher for Android test suite?

I missed it, I definitely like the changes to the test suite. I will give it a try sometime next week and let you know if I find anything else.

An extra point is that I wish you guys would consider supporting a version that something like BoringSSL which supports 64-bit builds out of the box.

@developernotes
Copy link
Member

I missed it, I definitely like the changes to the test suite. I will give it a try sometime next week and let you know if I find anything else.

Sounds good.

An extra point is that I wish you guys would consider supporting a version that something like BoringSSL which supports 64-bit builds out of the box.

The 1.1.0 release of OpenSSL, currently scheduled for release on May 12th, 2016 will include support for x64.

@brody4hire
Copy link

I just ran the test suite and it seems to run faster than before. Assuming that OpenSSL 1.1.0 is available next week will you release this change first or wait to include OpenSSL 1.1.0? (I would favor waiting for OpenSSL 1.1.0.)

@developernotes
Copy link
Member

Hi @brodybits

Yes, we would like to include the OpenSSL 1.1.0 release in a future beta build. We would like to gather ample feedback from the community regarding these changes before we integrate them forward as a public release.

@zokipirlo
Copy link

Internal testing looks fine. I released an app update on Play Store with this version of Sqlcipher :) Will tell you soon if any crash reports will occur.

@Pratyul
Copy link

Pratyul commented May 26, 2016

@developernotes
Have tried the android-n-preview branch on Android N and even previous OS versions. Have not seen any major issues or crashes so far. We wanted to plan Android N release and sql cipher would be an important part of that.
Even an approximate date like 1 month, 2 months will be good enough for the plan.

@ghost
Copy link

ghost commented Jun 2, 2016

I am using this code-branch on Android 5.1.1 and encountered a crash cannot reproduce.
` --------- beginning of crash
06-02 14:29:07.107 A/libc: Fatal signal 11 (SIGSEGV), code 2, fault addr 0xb7e43330 in tid 28485 (JDWP)

06-02 14:29:24.558 W/linker: libsqlcipher.so: unused DT entry: type 0x6ffffffe arg 0x1f5f8

06-02 14:29:24.558 W/linker: libsqlcipher.so: unused DT entry: type 0x6fffffff arg 0x3`
maybe someone saw this before? really hope some could help me with this, it took me the whole afternoon without any chance to figure it out..

@developernotes
Copy link
Member

Hello @lena8912

Is it correct that you only received this error once, you haven't been able to reproduce it again? What device did you experience this on, or was it on an emulator?

@ghost
Copy link

ghost commented Jun 2, 2016

Yes I receive this error once and not able to reproduce it again. I'm using lg nexus 4, it's a real device and running 5.1.1 and I used android-n-preview branch code.

@developernotes
Copy link
Member

Hello @lena8912

If you find you are able to reproduce this again, it would be beneficial to build the binaries locally and symbolicate the native stack trace from the adb logs. You can do this by executing the following before you run your application:

adb logcat | $NDK/ndk-stack -sym $PROJECT_PATH/obj/local/armeabi

The above $NDK references the root directory of your NDK folder and the $PROJECT_PATH is the root folder of the SQLCipher for Android source tree (following a build). The above is also relative to utilization of the armeabi binaries, adjust according to your target platform.

@ghost
Copy link

ghost commented Jun 3, 2016

Hello @developernotes
I haven't reproduced the crash yet, but I followed your instruction anyway.
I got the following:

$ adb logcat | $ANDROID_NDK_ROOT/ndk-stack -sym /obj/local/armeabi

********** Crash dump: **********
Build fingerprint: 'google/occam/mako:5.1/LMY47O/1783956:user/release-keys'
pid: 28475, tid: 28485, name: JDWP  >>> [packagename here] <<<
signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xb7e43330
Stack frame #00 pc 000a1330  [heap]
Stack frame #01 pc 00137ba1  /system/lib/libart.so (art::gc::Heap::GetObjectsAllocated() const+28)
Stack frame #02 pc 000f08bd  /system/lib/libart.so (art::Dbg::DdmSendHeapInfo(art::Dbg::HpifWhen)+912)
Stack frame #03 pc 000f0c83  /system/lib/libart.so (art::Dbg::DdmHandleHpifChunk(art::Dbg::HpifWhen)+114)
Stack frame #04 pc 0001a5e9  /data/dalvik-cache/arm/system@[email protected]
Crash dump is completed

********** Crash dump: **********
Build fingerprint: 'google/occam/mako:5.1/LMY47O/1783956:user/release-keys'
pid: 28672, tid: 28682, name: JDWP  >>> [packagename here] <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x550004
Stack frame #00 pc 00137b9e  /system/lib/libart.so (art::gc::Heap::GetObjectsAllocated() const+25)
Stack frame #01 pc 000f08bd  /system/lib/libart.so (art::Dbg::DdmSendHeapInfo(art::Dbg::HpifWhen)+912)
Stack frame #02 pc 000f0c83  /system/lib/libart.so (art::Dbg::DdmHandleHpifChunk(art::Dbg::HpifWhen)+114)
Stack frame #03 pc 0001a5e9  /data/dalvik-cache/arm/system@[email protected]
Crash dump is completed

********** Crash dump: **********
Build fingerprint: 'google/occam/mako:5.1/LMY47O/1783956:user/release-keys'
pid: 1573, tid: 1583, name: JDWP  >>> [packagename here] <<<
signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xb7d9d2a0
Stack frame #00 pc 000002a0  [heap]
Stack frame #01 pc 00137ba1  /system/lib/libart.so (art::gc::Heap::GetObjectsAllocated() const+28)
Stack frame #02 pc 000f08bd  /system/lib/libart.so (art::Dbg::DdmSendHeapInfo(art::Dbg::HpifWhen)+912)
Stack frame #03 pc 000f0c83  /system/lib/libart.so (art::Dbg::DdmHandleHpifChunk(art::Dbg::HpifWhen)+114)
Stack frame #04 pc 0001a5e9  /data/dalvik-cache/arm/system@[email protected]

since the first crash dump's : pid: 28475, tid: 28485, name: JDWP
is the same with my crash's tid = 28485:

 --------- beginning of crash
06-02 14:29:07.107 A/libc: Fatal signal 11 (SIGSEGV), code 2, fault addr 0xb7e43330 in tid 28485 (JDWP)

Is this information useful? Thanks for ur reply!!

@developernotes
Copy link
Member

Hi @lena8912

Thanks for looking into this further. The crash appears to be sourced within libc, and your log above references a linker warning for libsqlcipher.so. Do you still see the linker warnings even if there is no crash? Thanks

@ghost
Copy link

ghost commented Jun 3, 2016

@developernotes
Yes, there is a log whether crash or not. It appears every time I install and run my app(so I think it is not the reason why my app crashed). see below:

06-02 14:29:24.558 W/linker: libsqlcipher.so: unused DT entry: type 0x6ffffffe arg 0x1f5f8
06-02 14:29:24.558 W/linker: libsqlcipher.so: unused DT entry: type 0x6fffffff arg 0x3`

it is shown in yellow color in logcat, so I think it might be just a warning.
Any clues?!

@jaredpetker
Copy link

jaredpetker commented Jun 18, 2016

Building and usage on the x86 emulator works for me, however, building onto my Nexus 5x on newest Android M seems to produce this:

 java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "__aeabi_memcpy4" referenced by "/data/app/tigerconnect.app-2/lib/arm/libsqlcipher.so"...
                                                                      at java.lang.Runtime.loadLibrary(Runtime.java:372)
                                                                      at java.lang.System.loadLibrary(System.java:1076)
                                                                      at net.sqlcipher.database.SQLiteDatabase.loadLibs(SourceFile:177)
                                                                      at net.sqlcipher.database.SQLiteDatabase.loadLibs(SourceFile:170)

Issues with loading the sqlcipher.so it seems, any reason it can't find the symbol that I'm missing? For some more information, I was building with the newest ndk and targeting the android N final release for my application

@developernotes
Copy link
Member

Hi @jaredpetker

What branch of SQLCipher for Android are you compiling? What is the version number of your Android NDK? Are you running Android "M" (i.e., Marshmallow), or Android "N", the current beta OS on your Nexus 5X?

@jaredpetker
Copy link

@developernotes So it seems like the issue has to do with compiling sqlcipher with the newest (r12) ndk, using r11c seems to work fine (across x86 and arm for me). Seems to be an issue only for arm devices as my x86 emulator worked fine with the sqlcipher compiled with r12.

@developernotes
Copy link
Member

Hi @jaredpetker

Thanks for the quick response, I'm glad to hear that the issue was due to the NDK toolchain (unfortunately that is). Our building instructions suggest using r11c, glad to hear that is working well for you, too!

@jaredpetker
Copy link

Yea, I saw that initially and decided to live on the wild side which sadly bit me back, I assume you all have some pipeline internally for worrying about building with newer toolchains, until then yea, seems like all is well thanks!

@ashishmody
Copy link

Hey @developernotes, do we know when will the update with this fix be released? Do we have a timeline in place? Looks like Google froze the APIs for N and is encouraging developers to get their apps ready for N soon. Any timeline would really help us plan! Thanks!

@smarkovik
Copy link

I second that!

On Tue, Jun 21, 2016, 08:10 ashishmody [email protected] wrote:

Hey @developernotes https://github.com/developernotes, do we know when
will the update with this fix be released? Do we have a timeline in place?
Looks like Google froze the APIs for N and is encouraging developers to get
their apps ready for N soon. Any timeline would really help us plan! Thanks!


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#216 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AA8OPaFtx3G2bo27ZlI2z4Dkipaasx3lks5qN4A7gaJpZM4Htgon
.

Tancho

@developernotes
Copy link
Member

Hi @ashishmody, @smarkovik

We are planning a release soon, now that the N API has been finalized, and we've tested the library against the latest Developer Preview 4 (i.e., NPD56N). We appreciate your patience and enthusiasm!

@developernotes
Copy link
Member

Hi folks,

This is just an update, we have just release SQLCipher for Android 3.5.0 with support for Android N. More information can be found here. Thanks!

@dfyx
Copy link

dfyx commented Jun 24, 2016

Will there be official binaries? The link in https://www.zetetic.net/sqlcipher/sqlcipher-for-android/ doesn't work for anything newer than 3.1.0 and when I build my own binaries, they are much larger (5 MB for armeabi and armeabi-v7a and 7 MB for x86) than the 1.6 MB mentioned in the blog post.

@dfyx
Copy link

dfyx commented Jun 24, 2016

Nevermind, found the aar at http://central.maven.org/maven2/net/zetetic/android-database-sqlcipher/3.5.0/android-database-sqlcipher-3.5.0.aar which is just a renamed zip file.

(I just need the .so files for a .NET project)

@developernotes
Copy link
Member

Hi @dfyx

The native libraries included in the AAR package are compiled for the Android OS specifically and will not run on another environment.

@dfyx
Copy link

dfyx commented Jun 24, 2016

@developernotes I know. I was talking about a Xamarin Android project.

@CoconutByte
Copy link

CoconutByte commented Jul 1, 2016

@developernotes
when the column is defined by TEXT, It seems that the cursor.getInt() method can't return the right value of the column. Using the newest Version of sqlcipher released for Android N.

@developernotes
Copy link
Member

Hi @liuzhihuinb

I was not able to reproduce the problem you describe with the SQLCipher for Android test suite. Are you doing something different?

@CoconutByte
Copy link

CoconutByte commented Jul 4, 2016

@developernotes
Thank you for you reply.
I found the problem in my app when I updated the sqlCihper to the version of 3.5.1.
As my app worked perfectly with the sqlCihper before,I'm confused.
I have wrote a demo for using cursor.getInt() ,I think you can reproduce it with the code below.
https://github.com/liuzhihuinb/SqlCipherDemo.git

@developernotes
Copy link
Member

Hi @liuzhihuinb

It appears your application may have a problem with the min function instead? Are you able to reproduce the issue with the SQLCipher for Android test suite? We have a test that uses TEXT as the storage type, but requests an integer back from the cursor correctly..

@CoconutByte
Copy link

CoconutByte commented Jul 6, 2016

Hi @developernotes
Yes, I had reproduced this problem in the SQLCipher for android test suite.
As this issue had been closed. I submitted a new issue below since other people may encounter the same issue.

#244

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests