• After 15+ years, we've made a big change: Android Forums is now Early Bird Club. Learn more here.

Root Proposal/Theory for External SD storage solution on F6

WarrantyVoider said:
My guess is that not all files were copied successfully, since you've had problems with that part the first time. So like I said before, if
Code: 
cp -a /data/. $TMP_DIR
misses some things, maybe try
Code: 
tar -cp -C /data ./ | tar -xp -C $TMP_DIR
instead.

Another possibility is some cached files became corrupt when you went from booting with internal storage to booting with external SD. If so, cache and dalvik cache may need to be wiped so they could be rebuilt.

Last one I can think of right now is the file system may have an error. If that error is on the ext4 partition of the external SD, you can put the SD card in a Linux PC and run fsck to fix it. Or you can reformat that partition and start over from the state of your internal /data if you don't care about the data already on the SD.

Good luck.
Click to expand...

WarrantyVoider, I registered to thank you very much for your detailed instructions on how to link the SD card - it works!! I softbricked it once by installing both EZ file and FX file manager... but that's about the only hitch. Installed Assassin's Creed and Hitman GO and both seem to run fine so far *fingers crossed*

I used the above copy method with tar by copy and pasting over that line and everything seemed to copy over.

This method is much better than the Link2SD - I tried letting the battery drain to zero and all my apps and settings "stuck"

Thanks again!!!!
 
Did a bit more work. I made my own app_process, hoping that it would be more universally applicable. In JVene's implementation, a line to check and run zinit.sh is added to the app_main.cpp before zygote is loaded. Each time app_main.cpp is changed, as sometimes in a new Android version or with an add-on like Xposed, the source needs to be recompiled. So I hope this new implementation can avoid the need to do that. But then even if it works in theory, it doesn't mean it would work in practice. So if anyone knowledgeable enough to recover in case something doesn't work is willing to do some testing, I'd welcome any feedback, especially on KK roms or with Xposed.

Check my previous instructions for the previous implementation (post #179). Installation for this one is similar:

  1. Put "app_process-DataOnSD" in "/system/bin".
  2. Change its file owner and permissions to uid=0 (root), gid=2000 (shell), permissions=755 (-rwxr-xr-x).
  3. Rename the original "app_process" to "app_process-original" and rename "app_process-DataOnSD" to "app_process". The names MUST match exactly, or the phone won't boot!
  4. Create directory "/system/etc/init.d" (uid=0, gid=0, permissions=755, drwxr-xr-x) if it doesn't exist.
  5. Put "zinit.sh" in "system/etc/init.d".
  6. Change its file owner and permissions to uid=0, gid=0, permissions=750 (-rwxr-x---).
A note about using this with Xposed, assuming it works. There are 3 different "app_process"es: the original rom version, Xposed's and mine. Xposed-Disabler-Recovery.zip expects "/system/bin/app_process.orig" to be the original "app_process", whereas this implementation expects "/system/bin/app_process-original" to be the original. So, that means the name "app_process.orig" should really be my "app_process" file (i.e. "app_process-DataOnSD"), and the name "app_process-original" should really be Xposed's "app_process" file. The actual original rom version of "app_process" would be a backup and could be named something else like "app_process.rom-original" (rename that back to "app_process" to stop using both Xposed and my hack). I hope that makes sense.

To use or to uninstall, refer to the previous instructions.

If this works for everyone, hopefully we can solve the storage problem once and for all. Good luck.
 

Attachments

so I tried to install on hroark13's kk 442 beta 4...after re-boot, his boot img would start, and re- boot, so it boot looped. I did wait for about 10 mins to make sure, no dice.
 
WarrantyVoider said:
Did a bit more work. I made my own app_process, hoping that it would be more universally applicable. In JVene's implementation, a line to check and run zinit.sh is added to the app_main.cpp before zygote is loaded. Each time app_main.cpp is changed, as sometimes in a new Android version or with an add-on like Xposed, the source needs to be recompiled. So I hope this new implementation can avoid the need to do that. But then even if it works in theory, it doesn't mean it would work in practice. So if anyone knowledgeable enough to recover in case something doesn't work is willing to do some testing, I'd welcome any feedback, especially on KK roms or with Xposed.

Check my previous instructions for the previous implementation (post #179). Installation for this one is similar:

  1. Put "app_process-DataOnSD" in "/system/bin".
  2. Change its file owner and permissions to uid=0 (root), gid=2000 (shell), permissions=755 (-rwxr-xr-x).
  3. Rename the original "app_process" to "app_process-original" and rename "app_process-DataOnSD" to "app_process". The names MUST match exactly, or the phone won't boot!
  4. Create directory "/system/etc/init.d" (uid=0, gid=0, permissions=755, drwxr-xr-x) if it doesn't exist.
  5. Put "zinit.sh" in "system/etc/init.d".
  6. Change its file owner and permissions to uid=0, gid=0, permissions=750 (-rwxr-x---).
A note about using this with Xposed, assuming it works. There are 3 different "app_process"es: the original rom version, Xposed's and mine. Xposed-Disabler-Recovery.zip expects "/system/bin/app_process.orig" to be the original "app_process", whereas this implementation expects "/system/bin/app_process-original" to be the original. So, that means the name "app_process.orig" should really be my "app_process" file (i.e. "app_process-DataOnSD"), and the name "app_process-original" should really be Xposed's "app_process" file. The actual original rom version of "app_process" would be a backup and could be named something else like "app_process.rom-original" (rename that back to "app_process" to stop using both Xposed and my hack). I hope that makes sense.

To use or to uninstall, refer to the previous instructions.

If this works for everyone, hopefully we can solve the storage problem once and for all. Good luck.
Click to expand...

I have your old version installed on Xperion rom (xposed uninstalled before). I made a nandroid.
How can I update to your new version ?
 
Warrantyvoider, thanks for the step by step guide, it works! I soft bricked with boot loop by installing two different file managers but lg mobile support tool + emergency mode resolved it.

I have a quick question though - does accelerometer and magnet sensor still work for everyone? Everything works as if I have a new phone but this... if I remove the sd card then with the original it works, any suggestions? I installed sensor tests and it shows that the phone had Bosch sensor but it seems as if it's turned off... thanks.
 
crm701 said:
so I tried to install on hroark13's kk 442 beta 4...after re-boot, his boot img would start, and re- boot, so it boot looped. I did wait for about 10 mins to make sure, no dice.
Click to expand...
Thanks for testing and the report. There are a couple things for me to try. Unfortunately, I won't have access to a 2nd phone starting today. Since I need my own phone to work for the next couple days, I won't be able to flash roms for testing until later. Once I got my own implementation working on stock yesterday, I was hoping that I wouldn't have to wait a few days to find out whether it'd work in other cases so I posted the files. Well, now I know. Thanks. I got an old Nook Color I can use in the meantime for testing, but I'll need to update it to CM11 first.

aaanadie said:
I have your old version installed on Xperion rom (xposed uninstalled before). I made a nandroid.
How can I update to your new version ?
Click to expand...
The new implementation doesn't add any functionality. You don't need to update. But if you want to help me test this implementation on Xperion without Xposed, just do "To uninstall" in post #179. Then do the installation steps in post #202. After reboot, if the phone is using the external SD for /data, then it's working.

If you want to add Xposed to Xperion and understand what I meant in post #202, well, it can be done manually. The problem is if you use Xposed to disable/enable itself, it can change the "app_process" file and this SD hack could be disabled as well. So it's a bit complicated to get everything to work as intended. If this is what you want to do, I can look into doing checks and making changes in the script so as to be not too confusing. But that would have to wait until I can do testing on my phone.

P.S.
Well, now thinking about this problem some more, perhaps it was a bit too premature to offer this thing for testing. A lot more testing needs to be done to make this concept a solution that would work with many different configurations.
 
WarrantyVoider said:
Did a bit more work. I made my own app_process, hoping that it would be more universally applicable...
Click to expand...

Got this to work on rooted stock rom, software version D50010b.

First time I did it, I renamed the original app_process to app_process_original, instead of app_process-original. It bootlooped and I got to learn a little bit about using adb and changing my underwear.

Requesting inclusion of .LGF6DataOnSD_INIT in your archive, just because win8 busts my balls when I try to rename a file to have a single period at the front of the file name.

Say I remove .LGF6DataOnSD_DO_NOT_REMOVE from data, and put LGF6DataOnSD_INIT back into the fat32 partition. Does it wipe the ext partition first, or does it just copy back in, overwriting things that already exist?
 
Hmm... I used a hyphen to connect "app_process" and "original" because the underscore is used for the original filename. I tried to use a different character so that it'd be easier to tell (for me at least) which part is the original name and which part is a suffix. I could've used a dot, but that's normally used to denote the file extension/type. Sorry to cause the confusion.

I probably don't need a dot in front of the ".LGF6DataOnSD_INIT" but I was trying to be consistent with the other filename. ".LGF6DataOnSD_DO_NOT_REMOVE" stays in /data so I want to make it a hidden dot file. I figured ".LGF6DataOnSD_INIT" can be created on the phone directly. So, I don't think I'll include that file in the archive, but next time I'll go with "LGF6DataOnSD_INIT" and ".LGF6DataOnSD_DO_NOT_REMOVE", then they would be more compatible with the partitions (fat32 and ext4, respectively) in which they belong.

Lastly, using the "INIT" trigger will remove all files before copying. It's meant to be a way to start over. It'd be like a data reset, except going back to a known state from internal storage rather than all the way back to the factory condition. I think this use case makes more sense. One can make a script to update the file system instead, but it's probably a lot trickier. In the script, I only remove files. Formatting is probably better, but I'm trying to rely only on the tools already provided by stock Android. Maybe there's a way, but I haven't looked into it. If formatting is needed to fix bad blocks or other errors on the SD, we can just do so on a PC so I'm not concerned about it.

I'm glad you got this to work. Thanks for letting me know.
 
OK, an update for those who have been following my development. I still can't have my phone offline yet, so I'm using an old Nook Color for testing. Nook Color allows me to run the system completely from an external microSD card. So I loaded CM11 (running Android 4.4.4), installed my app_process (as in post #202), and rebooted. No boot issues. Then I modified the zinit.sh to use the corresponding partition locations and installed it into /system/etc/init.d (CM11 already has the directory). On my SD card, I have a blank partition larger than the one already used by the system as /data. I placed the "INIT" tag file at the right location, and rebooted. As expected, booting up took a while to copy files from the previous /data partition to the blank partition. Afterwards, the system booted up with the larger partition as /data. Android didn't complain or give me any errors. So in conclusion, the implementation (script edited to point to the right partition locations) worked in a completely different platform, running KK.

I think the implementation should work with Xposed as well, but since Xposed also installs a modified app_process, getting them to coexist will take some effort. Like I said before, they would probably work right now, as long as Xposed doesn't try to enable/reinstall/disable/uninstall itself (thus modifying app_process again).

If anyone can confirm my implementation works for them running a different rom or Android version, I'll then focus my attention to making the installation more foolproof. In the meantime, I'll see about doing more testing with other roms/systems/devices.
 
a8888a said:
I have a quick question though - does accelerometer and magnet sensor still work for everyone? Everything works as if I have a new phone but this... if I remove the sd card then with the original it works, any suggestions? I installed sensor tests and it shows that the phone had Bosch sensor but it seems as if it's turned off... thanks.
Click to expand...
Somehow your posts didn't show up for me previously, so I only saw them today. Anyway, you're right. Thanks for discovering that. The sensors are indeed not working. I just checked this out. I think the reason they are not working is because by the time the script runs, the sensor daemon (/system/bin/sensord) is already accessing the files in /data/misc/sensor. I noticed before I couldn't unmount the /data partition, so I just mount the external one over the /data mount point. Everything else works fine because other than the files in /data/misc/sensor, no other files in /data seem to be open at that time.

I recall JVene wanted to implement what the script does in C/C++, maybe this is why. Right now, I see two potential ways to deal with this. One, implement in C like JVene planned and see whether the faster compiled code gets /data mounted before sensord runs. One potential problem is if the external storage system is not yet ready before sensord wants to access /data, then this method is no good. The other possibility is to block sensord until the new /data partition is mounted, since the sensors probably aren't needed for booting. But this method seems like a hack (on top of this SD hack), so it might not be easily applicable to other platforms. If anyone has any other ideas, please share.

Of course, there's also the possibility of not moving /data/misc or other hardware related directories, i.e. moving everything else. I think other external storage solutions take this approach, using symlinks (Link2SD) or bind mounts (Mounts2SD).

a8888a said:
This method is much better than the Link2SD
Click to expand...
I think so too, so I want to try to get it to work. Previously crm701 mentioned Mounts2SD. I think it used to have something called R-Mount which mounts external partition as /data but that feature was removed because of too many issues with it. There's also an old script called D2EXT which is implemented by CronMod, but CronMod seems to have discontinued that method as well.

Custom roms usually support init.d and busybox, so there are several options for the storage problem. I want to try to make JVene's method work for stock roms because the method would make very few changes to the system. I don't know if I would succeed, but there's much to experiment.
 
WarrantyVoider said:
Somehow your posts didn't show up for me previously, so I only saw them today. Anyway, you're right. Thanks for discovering that. The sensors are indeed not working...
Click to expand...

Just discovered this myself and came to post about it. Kindof funny we didn't notice it sooner. Shows how often we try to rotate our screens, I guess.

Disclaimer: I am not a Linux guy

Could you do something like...
1. Phone starts to boot as usual, sensor accesses /data/misc/sensor as usual, before you do your magic
2. Create an alternate link or path or to /data/misc/sensor or give it a different name or something
3. Do your data remount
4. Remount the alternate path as /data/misc/sensor

Or am I just making shit up? :p
 
Hehehe I fixed it!

But I can't tell you how, because I don't know how I did it! lolol

Here is what I know I did:
I ran FX
Navigated to /system/bin/
Took turns executing sensord, sensors.qcom, and sensorservice
Tried mucking about with my phone and a sensor tester app, with NO LUCK
Put phone down while in sensor app. Screen timed out while I played on laptop.
Picked up phone, and when I did my unlock screen the sensor app showed my XYZ accelerometer data! My screen rotated, angels played trumpets, etc etc.

Now make sure you repeat those steps exactly or else it won't work ; )

I have not tried to reboot and recreate this. Maybe I'll give it a shot in a little bit.
 
I actually ran into this problem a while ago, but thought it was just me. Running "sensord &" as root then locking and unlocking the phone works for me.
 
I logged on earlier today and saw the problem. I log on tonight and see the workaround. Good job, bigfuzzymonster and Lhy19! Just to clarify, for those who use my script, there are two "mount... /data" lines in the script:
Code: 
mount -t ext4 -o $MOUNT_OPT $PARTITION2 /data
One is for normal mounting and the other is for mounting after initialization (file copying). So you would run "sensord" after each situation:
Code: 
mount -t ext4 -o $MOUNT_OPT $PARTITION2 /data
/system/bin/sensord &
There would be two instances of sensord running, but I assume doing so doesn't affect sensor operations. I'll still try to implement a cleaner solution, but at least the current solution is usable.

Edit: like likwid2 stated below (#234), using stop/start also fixes the sensors (maybe a cleaner solution):
Code: 
stop sensord
mount -t ext4 -o $MOUNT_OPT $PARTITION2 /data
start sensord
 
I'll have updated instructions when I post my next release. My implementation is intended for stock roms because stock roms don't have init.d support or busybox. For custom roms with init.d and busybox, there are other better supported and more tested options like Mounts2SD. Xperion supposedly uses the stock rom as a base, so some have reported success with my implementation. I do have my implementation working with CM11 on a Nook Color, so I'm inclined to think it could work in general with other custom roms as well. But so far at least two people have tried it with KK roms and reported boot issues. At this point I don't know if those boot problems are caused by installation errors or incompatibilities. Since several people have run into boot issues because of installation mistakes, I'm working on an installation script that would hopefully make things a bit less error-prone.

Right now I'm trying to get it to work with Xposed. I could get it to work before with a tweak here and there, but I want both working without modification so updates won't cause problems. With the new version, I have both running as they should on the Nook Color. So hopefully that problem is solved and I can move on to the next thing on my to-do list.

Anyway, if you or anyone else is willing to be a guinea pig and help development by testing (or even help development by developing) so that I don't have to spend a lot time on that part, I'll make a release soon. If not, that's all right. I should eventually have something for those who want to solve their storage problem with this method.
 
WarrantyVoider said:
I'll have updated instructions when I post my next release. My implementation is intended for stock roms because stock roms don't have init.d support or busybox. For custom roms with init.d and busybox, there are other better supported and more tested options like Mounts2SD. Xperion supposedly uses the stock rom as a base, so some have reported success with my implementation. I do have my implementation working with CM11 on a Nook Color, so I'm inclined to think it could work in general with other custom roms as well. But so far at least two people have tried it with KK roms and reported boot issues. At this point I don't know if those boot problems are caused by installation errors or incompatibilities. Since several people have run into boot issues because of installation mistakes, I'm working on an installation script that would hopefully make things a bit less error-prone.

Right now I'm trying to get it to work with Xposed. I could get it to work before with a tweak here and there, but I want both working without modification so updates won't cause problems. With the new version, I have both running as they should on the Nook Color. So hopefully that problem is solved and I can move on to the next thing on my to-do list.

Anyway, if you or anyone else is willing to be a guinea pig and help development by testing (or even help development by developing) so that I don't have to spend a lot time on that part, I'll make a release soon. If not, that's all right. I should eventually have something for those who want to solve their storage problem with this method.
Click to expand...

I am willing to be one of your guinea pigs but I don't know what I could do in dev department.

Thanks for the work you are doing for a solution for this sad limitation of storage issue.
 
All right, as promised, I'm posting my current version for testing. If anyone is willing to test this and give me feedback (does everything work? problems you've encountered? what roms have you tried?), follow the brief instructions. Make sure the thing is installed correctly before reporting it doesn't work. But if you run into problems, let me know too so I know what to look out for when I make an installation script. Pay attention to the letter case and special characters. If anyone is not trying to test this, then wait for a release with an installation script or with better instructions.

Inside the zip are an executable file named "script_caller" (size=9496) and a directory "DataOnSD" containing two shell scripts, named "debuggerd" (size=1169) and "sensord" (size=148).

First install the binary file:

  1. Copy "script_caller" to /system/bin .
  2. Fix its owner/permissions: uid=0 (root), gid=2000 (shell), permissions=755 (-rwxr-xr-x).
  3. In /system/bin , rename "debuggerd" to "debuggerd-original" (a hyphen, not an underscore). Make sure names are right!
  4. In /system/bin , rename "script_caller" to "debuggerd".
  5. Copy "script_caller" from the zip to /system/bin (Yes, same file as in step 1).
  6. Fix its owner/permissions as in step 2: uid=0, gid=2000, permissions=755.
  7. In /system/bin , rename "sensord" to "sensord-original" (again, hyphen). Check names!
  8. In /system/bin , rename "script_caller" to "sensord".
OK, now the binaries are installed. Next, install the scripts:

  1. Copy the directory and its two files to /system/etc .
  2. Fix their owners/permissions:
    • /system/etc/DataOnSD : uid=0, gid=0, permissions=755 (drwxr-xr-x).
    • /system/etc/DataOnSD/debuggerd : uid=gid=0, permissions=700 (-rwx------).
    • /system/etc/DataOnSD/sensord : uid=gid=0, permissions=700 (-rwx------).
Next, prepare the SD card. It's best to start with a blank card. If you've already set up the SD card with my script before, there shouldn't be a need to do anything to the SD card. So reboot and hope the system boots correctly and everything works. If the SD has not been set up yet, use a card with fast random reading/writing performance like Samsung or SanDisk and do the following:

  1. Create two partitions: 1st=fat32 and 2nd=ext4. Use modern partitioning software to ensure partitions are properly aligned.
  2. Make an empty file "LGF6DataOnSD_INIT" in the root directory of the fat32 partition.
  3. Put SD card into the phone.
If everything checks out so far, reboot the phone and allow the scripts to do their thing. First boot will take a while depending on how much data need to be copied. All files and settings should carry over from the internal storage to the external SD. Let me know if something doesn't get copied correctly.

Once I know this implementation works for sure, I'll make a post with technical details in case someone else wants to understand and check my approach. Good luck and I look forward to getting feedback.
 

Attachments

Just wondering still put the period in front of LGF6DataOnSD_INIT? I am assuming yes but want to be positive about it.


If all goes correctly, should display the EXT4 (partition 2) as Android's internal storage under settings, storage? Or check another way?
 
No dot in front of "LGF6DataOnSD_INIT". In post #208 I said I would remove the dot because Windows users have a hard time creating that file on their computer. The file is removed as soon as my script sees it, so if for some reason file copying fails and the "INIT" step needs to be started again, "LGF6DataOnSD_INIT" would need to be recreated. Doing the step on the phone itself should pose no problem.

If all goes according to design, then the first thing would be the phone boots up correctly. That indicates the binary part did not trigger a boot loop. If the script part works, the phone would have increased storage, so yes, the internal storage under settings should show the capacity of the external ext4 partition. On a custom rom, there's still a possibility something else doesn't work because its "init" loads services in a different order or at a different pace. If that's the case, then we would have to determine what doesn't work and adapt the script for that case. On the stock rom, the sensors fail to work, so I delay its loading. That's the "sensord" part. It may not be necessary on a custom rom, but I include that step in my instructions just to eliminate that potential problem for the time being.
 
Gotcha...Ok so just for info purposes running on KK beta 4, used the kitkat "fix" which makes the sd card readablem and installed a zip which gives me a functional init.d (was using setup for mounts2sd) I installed your package per instructions. When I rebooted, the files from before on the EXT4 partition (the apps) appeared back in my apps drawer (in fact I watched them materialize one by one) so no bootloop, and it did something but under storage didnt see a change. Also my settings all went back to original.

I plan (at some point) on actually resetting the phone back to stock with the KDZ, installing KK again and testing out again... just passing this along... by the way dude thanks for not disappearing! and excellent work

forgot to mention that the home button and "q" button (top left) stopped working but I dont think that was a problem from your scripts.


Here's something that just floated to my attention...This is made to work on 4.1.2 right? So if the file structure is different in KK, would the scripts still do the flip? I was looking at some other forums with people doing the ext to int thing and some of the paths for 4.1.2 seem different...not sure if this is applicable. I'm Not really on with the scripting, just wondering...
 
so like I said I wanted to try your work on a clean install, only tested before out of curiosity. After a clean flash, followed the steps and hey! it worked! very nice

so now under android storage I have 23 gb "internal" with 5 gb external...I have only tried download "walking dead"season 1 (over a gb download) it dl'ed, installed and played...all settings stuck, nothing vanished and sensors seem ok...going to mess around with it some more but awesome stuff!

info: clean install hroark13's kitkat beta 4 (masterchief lg mix), nothing else, your mod (not a hack lol)
 
You must log in or register to reply here.
Back
Top Bottom