SD hack for storage expansion

[WarrantyVoider]

DataOnSD

Introduction
This is a hack to allow the second partition of an SD card to be used as data storage space on an Android phone. The original idea is from JVene as discussed in this thread:
http://androidforums.com/threads/proposal-theory-for-external-sd-storage-solution-on-f6.819792/
The current implementation has evolved from the original proposal. For more information, please check the original thread.

Disclaimer
No one but you is responsible for what you do to your device. There is no guarantee of any kind whatsoever that any files would work as claimed. There is no guarantee of any kind whatsoever that any information would even be safe to apply. Proceed at your own risk!

Prerequisites

• Capable microSD card (with fast random reading and writing like Samsung or SanDisk)
  
• Custom recovery to install flashable zips (either CWM or TWRP for Jelly Bean roms, TWRP for KitKat roms)
  
• Functional rom/system (set up device before applying the hack)

Spoiler: Click here for more info

Back up your files and data before making changes!

Root
To install a custom recovery, we first need to root the phone.
See this thread for more info:
http://androidforums.com/threads/guide-rooting-the-optimus-f6-and-recovery.823131/
You can uninstall tr.apk afterwards if you wish. Once you're rooted, the apk itself isn't needed.
You can optionally install BusyBox if you wish. Some root apps require it, but it is not needed by this hack except the init.d version. Most if not all roms that support init.d also have BusyBox already built-in.
Root is needed to install custom recovery. Custom recovery is used to install the hack. Usage of the hack should not require root, so if desired, it should be possible to unroot after the hack is installed and functioning.

Recovery
For recovery, install hroark13's TWRP:
http://androidforums.com/threads/recovery-twrp-lokied-new-apk-version-08-27-14.861281/
Install the apk. Open app. Install recovery. (BusyBox might be needed for this.)
You can keep the app to reboot to recovery or you can install a different one like GameTheory's app.

Functional rom
Set up the rom as you wish. When you install the hack, all files in the internal /data folder (apps, settings, etc.) will be copied to the SD.
If you want to use a new rom, flash it first. Reboot into the system to set up the rom. Then go back into recovery without triggering factory reset and install the hack.

Installation

1. Partition the SD card: 1st partition=fat32, 2nd=ext4.
   
2. Boot into recovery without triggering data wipe.
   
3. Flash the appropriate zip file(s).
   • If rom has init.d support and BusyBox, you can try DataOnSD-initd.zip (but this is not as well tested/supported).
     
   • Otherwise, flash DataOnSD-copy.zip and then DataOnSD-install.zip.
4. Reboot into system and determine whether additional workarounds are necessary.

Spoiler: Click here for more info

Back up your files and data before making changes!

Partitioning and formatting
There are many ways to partition the card. On a Windows PC, you can use MiniTool Partition Manager. On the phone, there are apps like AParted. Remember to align partitions at least to MB to avoid misalignment causing poor performance and shortened lifespan. After the SD card is set up, copy the necessary files to the card.
I also provide DataOnSD-partition-NN.zip that can be used in recovery. If you decide to use it instead, read the info associated with the zip. Once you're in recovery, install the renamed DataOnSD-partition-NN.zip before installing DataOnSD-copy.zip.

Booting recovery
Do not use volume down + power on the F6 to reboot into recovery. Use a method that does not trigger a factory reset on this phone: TWRP app's reboot function, reboot app (e.g. GameTheory's app), custom rom's advanced reboot option, "reboot recovery" at a command prompt, or "adb reboot recovery" with ADB.
Once in recovery, it's highly recommended that you make a full backup before making system changes.

Wiping external partition
If you've just partitioned and formatted the SD card, this step is not necessary. However, if the ext4 partition on the SD is not empty, wipe it first.
Settings -> check "use rm -rf..." -> back to main screen -> Wipe -> Advanced Wipe -> check only "sd-ext" -> swipe to wipe -> back to main screen -> Settings -> uncheck "use rm -rf..." -> back to main.

Flashing zips
Install -> find location of your zip files -> select DataOnSD-copy.zip -> swipe to flash -> wait for file copying to finish and make sure no errors -> back -> select DataOnSD-install.zip -> swipe to flash -> Reboot System.

Checking functionality
Space: Obviously, if the hack works, your storage space should now be the size of your external ext4 partition.
Sensors: If they don't work, auto-rotate doesn't work. So if you have a problem with sensors, wait a couple of minutes to let the phone settle down after boot. Then reboot and check again. If the sensors still don't work, you need to apply the sensor workaround.
KK SD permission: If you're running a KitKat rom, download and install a new file manager that wasn't installed to the internal storage. If you can't or don't want to access the Play Store, you can try Total Commander: http://www.ghisler.com/android.htm
Without giving superuser privileges to the file manager, try to create a file or folder in the emulated SD. If that fails, you need to apply the KK-permission workaround.

Applying workarounds
Reboot to recovery and flash the appropriate zip: -sensorfix.zip for sensor workaround only, -kksdfix.zip for KK-permission workaround only, or -sensor-kksdfix.zip for both.

Usage Notes

• You need an SD card with adequate random read/write performance because the OS will be reading from and writing to the SD card frequently.
  
• This hack uses the ext4 partition on the SD card as the /data partition. Do not remove the SD card from the phone while the phone is on.
• Android's data encryption feature isn't supported. Since all of your user data is on the removable SD card, beware of the security implication.
  
• The /data partition in the internal storage should remain intact and hidden. If the hack is not active (e.g. hack is disabled, partition fails to mount, or SD card is removed), the phone should still boot using data from the internal storage.
• Sometimes the ext4 partition on the SD card fails to mount (because of forced reboot/shutdown, file system error, bad flash memory cell, etc.) and the phone boots with the internal /data storage instead. Reboot into recovery and flash -fsck-ondemand.zip to attempt to fix the partition.
• Some roms require additional workarounds. Flash the appropriate zips in recovery to use them if necessary.
• Some settings (e.g. USB debugging) are read and applied by Android before the hack is activated during boot-up. Changes to those settings get written to the SD card, but Android reads the values from the internal storage on next boot. To make those settings persistent, uninstall the hack, change the settings, and reinstall the hack/workaround (no need to copy).

Files

• Universal version (updated 4/02/2015)
  • -copy.zip
  • -install.zip
  • -uninstall.zip
• Init.d version (#319 of original thread)
  • -initd.zip
• Workarounds (updated 4/17/2015)
  • -sensorfix.zip
  • -kksdfix.zip
  • -sensor-kksdfix.zip
• Maintenance tools (updated 4/17/2015)
  • -fsck-ondemand.zip
  • -wipe-dalvik.zip
• Helper tool (updated 6/07/2015)
  • -partition-NN.zip
• Basic backup/restore (#460 and #476 of original thread)
  • -backup.zip
  • -restore.zip

Thanks

• JVene for the original idea and technical info.
• crm701 for testing and feedback, especially KK testing.
• stigmax2 for testing and feedback to help get it working in TWRP.
  
• likwid2, Lhy19, and bigfuzzymonster for ideas about sensor workaround.
• RelZeshi and orion2323 for reporting KK permission issue and helping test workaround.
• TrevorX5J9 for reporting and helping diagnose mounting issue caused by file system errors.
• Krlypumaa for reporting issue with Dalvik wiper and testing a fix.
  
• GameTheory for tips to package installation script into flashable zip.
• Everyone who has tried, tested, used, and/or provided feedback for this hack.
• Everyone who has contributed to the original thread.
• Anyone else I forget, please remind me.

 

[WarrantyVoider]

Changes:
Apr 2 - updated installer to reduce size and to provide better installation messages; no functional changes.
Initial - Zips from #319 of original thread

-copy.zip
Copies data files from internal storage to external partition. It does not modify system files nor internal data files.

-install.zip
Installs binary to the /system folder. It does not modify internal data files nor external SD partition.

-uninstall.zip
Uninstalls changes made by the installer. It removes both the hack's main binary and workaround script. It does not modify the external SD partition.

 

[KOLIO]

Stickied.

 

[cerebralcow]

Hi I'm new to rooting and ROMs. I just rooted my phone for the first time and the only hack I really need is something that will get around the low internal memory of the F6.
I'm just looking for a really simple solution that doesn't require me to reset my phone.
This is the easiest solution I have found but when I checked the CWM and TWRP websites I couldn't find a version for the F6. Could someone please tell me how I can find a custom recovery to install flashable zips

 

[GameTheory]

Hi I'm new to rooting and ROMs. I just rooted my phone for the first time and the only hack I really need is something that will get around the low internal memory of the F6.
I'm just looking for a really simple solution that doesn't require me to reset my phone.
This is the easiest solution I have found but when I checked the CWM and TWRP websites I couldn't find a version for the F6. Could someone please tell me how I can find a custom recovery to install flashable zips

Here is twrp recovery for the F6 located in the root section of the f6 forum...
http://androidforums.com/threads/recovery-twrp-lokied-new-apk-version-08-27-14.861281/

 

[aangbell]

Can you tell me if this microSD works for this hack?
I want to make sure if it works. Please.http://www.bestbuy.com/site/sandisk...lack/4120107.p?id=1218457270079&skuId=4120107

 

[cerebralcow]

Yes class 10 Ultra Micro SD cards have a good transfer rate. 30 megabytes per second (which is what that card has) is about the fastest you're going to get unless you want to pay extra just for speed.

 

[WarrantyVoider]

Can you tell me if this microSD works for this hack?

Yes. Any Samsung or SanDisk should work fine. I'm using a Samsung EVO class-10. I developed the hack with a Samsung class-6. I've also tested the hack with a SanDisk class-4 on an old Nook Color. Like cerebralcow says, a better, faster, and more expensive card could give you slightly better random r/w performance, much better sequential performance (helps with transferring videos and large photos, loading large games, etc.), and/or better reliability (MLC vs TLC), but SanDisk Ultra should be good enough for general use.

 

[aangbell]

Yes. Any Samsung or SanDisk should work fine.

Thank you! Another question that i have is if this hack works with the xperion rom specifically the last update, is deodexed version so i don't know if it affect or not.

 

[GameTheory]

Thank you! Another question that i have is if this hack works with the xperion rom specifically the last update, is deodexed version so i don't know if it affect or not.

Yes, it works with all the f6 roms. Deodex has nothing to do with the sd hack.

 

[healthylaff]

Bump! Any way someone would be kind enough to give a click-by-click (more detailed) instruction set for the OP "Installation" below? TIA

DataOnSD

Installation

1. Partition the SD card: 1st partition=fat32, 2nd=ext4.
   
2. Boot into recovery without triggering data wipe.
   
3. Flash the appropriate zip file(s).
   • If rom has init.d support, you can try DataOnSD-initd.zip (but this is not as well tested/supported).
     
   • Otherwise, flash DataOnSD-copy.zip and then DataOnSD-install.zip.
4. Reboot into system and determine whether additional workarounds are necessary.

Also, a few brief questions:

• I believe I installed the CWM recovery during my initial tweaking... how do I check to see if it's there and working properly?
• Currently I have Link2SD installed and lots of data defaulting there. Can I remove that microSD safely (using the instructions about properly unmounting it before removal), then format a new microSD, per the instructions with your http://androidforums.com/threads/sd-hack-for-storage-expansion.908126/ link and go from there, without doing anything else special?
• I'd certainly be willing to reinstall all of my apps with the new memory expansion goes well
• What would you recommend for backing everything up before starting? I have Titanium Backup installed but really haven't used it yet.

 

[aangbell]

Yes, it works with all the f6 roms. Deodex has nothing to do with the sd hack.

Thank you GT! I've already done this method on your ROM, thank you so much for your contribution to the people who want a better phone, and also to all developers who helped to do this. It's simply awesome!

 

[WarrantyVoider]

Bump! Any way someone would be kind enough to give a click-by-click (more detailed) instruction set for the OP "Installation" below? TIA



Also, a few brief questions:

• I believe I installed the CWM recovery during my initial tweaking... how do I check to see if it's there and working properly?
• Currently I have Link2SD installed and lots of data defaulting there. Can I remove that microSD safely (using the instructions about properly unmounting it before removal), then format a new microSD, per the instructions with your http://androidforums.com/threads/sd-hack-for-storage-expansion.908126/ link and go from there, without doing anything else special?
• I'd certainly be willing to reinstall all of my apps with the new memory expansion goes well
• What would you recommend for backing everything up before starting? I have Titanium Backup installed but really haven't used it yet.

Not quite click-by-click, but hopefully the added info is enough to get you going.

If you've installed CWM, you should be able to install a reboot app and boot into CWM. Once in CWM, Use the physical buttons to navigate.

I don't have experience with Link2SD, so maybe someone else can help you better. I don't know what exact system changes are made by Link2SD. If I do, I might be able to help you revert the changes. Any reason you're considering using this hack rather than continuing with Link2SD? I know what I would do in your situation, but the steps involved would sound way too technical and convoluted. Let me know if you actually want to go there.

As for backing up and restoring individual apps and settings, that's probably the easier way to go. Again, I'm not qualified to help you here though, so hopefully someone can give you the answers you need.

Let me know if you need my help. Good luck.

 

[Krlypumaa]

Not quite click-by-click, but hopefully the added info is enough to get you going.

If you've installed CWM, you should be able to install a reboot app and boot into CWM. Once in CWM, Use the physical buttons to navigate.

I don't have experience with Link2SD, so maybe someone else can help you better. I don't know what exact system changes are made by Link2SD. If I do, I might be able to help you revert the changes. Any reason you're considering using this hack rather than continuing with Link2SD? I know what I would do in your situation, but the steps involved would sound way too technical and convoluted. Let me know if you actually want to go there.

As for backing up and restoring individual apps and settings, that's probably the easier way to go. Again, I'm not qualified to help you here though, so hopefully someone can give you the answers you need.

Let me know if you need my help. Good luck.

Hi bro... With ur hack its no need to do link2sd.. I have used ur hack for Carbon.Xperion.KK.mini which i hav all three and run back and forward to help people with their issues... 1 thing though how can i make it survive when i wipe/dalvik... It seems sdhack disappear when i wipe within same same rom.

 

[WarrantyVoider]

@Krlypumaa
I'll investigate the issue later, but my guess is wiping dalvik deletes the folder as well. I think during boot the folder in internal /data gets recreated, but since that happens before the hack is activated, the external /data (i.e. ext4 partition on SD) is missing the dalvik folder. My hack checks for the existence of that folder before mounting the partition. I think when I made the decision to do that initially, I was trying to do a simple check to make sure the directory structure was intact. I didn't bother with checking all folders, just the dalvik folder. At the time I probably thought the other mods/hacks that symlink or bind-mount some folders would leave dalvik in the internal storage for speed, so an external partition containing that folder would more likely contain all folders copied by my hack. Later I started using a tag file in addition to be sure.

Your data should still be there. Until I have time to look into this issue and come up with a sensible solution, you should be able to manually add the folder back. In recovery, mount /sd-ext and create /sd-ext/dalvik-cache (which, once mounted by my hack, should become /data/dalvik-cache). I don't remember its permissions and selinux contexts off the top of my head; check the dalvik folder in the internal storage. Then do chown/chmod/chcon to fix the folder. I can give you the exact commands to type later if you need them.
Edit: OK, just looked it up. I'll probably make a zip to wipe dalvik for my hack and to fix/recreate folder.

Spoiler: ownership/permissions/context

From internal /data under KK:

~ # ls -dZ /data/dalvik-cache
drwxrwx--x system  system  u:object_r:dalvikcache_data_file:s0 dalvik-cache

So to fix, mount /data and /sd-ext in TWRP, issues commands in terminal:

chown system. /sd-ext/dalvik-cache
chmod 771 /sd-ext/dalvik-cache
busybox chcon --reference=/data/dalvik-cache /sd-ext/dalvik-cache

I know what a chore it can be to flash/restore to check some issue in another rom. So thanks for taking time to help other users. I've been working on something that could help you switch roms easily. I still have some work to do before I can release it though.

 

[Krlypumaa]

@Krlypumaa
I'll investigate the issue later, but my guess is wiping dalvik deletes the folder as well. I think during boot the folder in internal /data gets recreated, but since that happens before the hack is activated, the external /data (i.e. ext4 partition on SD) is missing the dalvik folder. My hack checks for the existence of that folder before mounting the partition. I think when I made the decision to do that initially, I was trying to do a simple check to make sure the directory structure was intact. I didn't bother with checking all folders, just the dalvik folder. At the time I probably thought the other mods/hacks that symlink or bind-mount some folders would leave dalvik in the internal storage for speed, so an external partition containing that folder would more likely contain all folders copied by my hack. Later I started using a tag file in addition to be sure.

Your data should still be there. Until I have time to look into this issue and come up with a sensible solution, you should be able to manually add the folder back. In recovery, mount /sd-ext and create /sd-ext/dalvik-cache (which, once mounted by my hack, should become /data/dalvik-cache). I don't remember its permissions and selinux contexts off the top of my head; check the dalvik folder in the internal storage. Then do chown/chmod/chcon to fix the folder. I can give you the exact commands to type later if you need them.
Edit: OK, just looked it up. I'll probably make a zip to wipe dalvik for my hack and to fix/recreate folder.

Spoiler: ownership/permissions/context

From internal /data under KK:

~ # ls -dZ /data/dalvik-cache
drwxrwx--x system  system  u:object_r:dalvikcache_data_file:s0 dalvik-cache

So to fix, mount /data and /sd-ext in TWRP, issues commands in terminal:

chown system. /sd-ext/dalvik-cache
chmod 771 /sd-ext/dalvik-cache
busybox chcon --reference=/data/dalvik-cache /sd-ext/dalvik-cache

I know what a chore it can be to flash/restore to check some issue in another rom. So thanks for taking time to help other users. I've been working on something that could help you switch roms easily. I still have some work to do before I can release it though.

Thank you very much for taking ur time on this.. I haved check the folders have to say that u are correct. Take the time u need... Just so u knw what ever that u need to test send it over... I won't mind by guinea pig in yo test... Make sure u send it to me in PM. Inbox.. So we can make sure when realising it it wont bring so much ???? To our fellas... Good looking out.

 

[Krlypumaa]

Do u think im mounting everything in the sdext... I think that may be the issues.. Not sure

 

[WarrantyVoider]

I'll probably make a zip to wipe dalvik for my hack and to fix/recreate folder.

OK, here's the flashable zip. I hope this is enough to deal with the issue.

The wipe-dalvik function of a custom recovery wipes three folders: /data/dalvik-cache (internal /data), /cache/dalvik-cache (location used by CM roms), and /sd-ext/dalvik-cache (2nd partition of SD used by the hack for data). This zip wipes /sd-ext/dalvik-cache and /cache/dalvik-cache, but leaves /data/dalvik-cache alone. So this zip could be used to replace the wipe-dalvik function for users of my hack. As I said before, using the built-in wipe-dalvik also deletes the dalvik-cache folder itself. Without that folder, the hack wouldn't mount the partition. Flashing the zip when that folder is missing would recreate the missing folder, so the zip could be used after doing the wipe-dalvik function to fix the issue of the missing folder.
Note: for KK roms, this should to be flashed with a recovery that supports selinux. So on the F6, use TWRP.

 

[Krlypumaa]

OK, here's the flashable zip. I hope this is enough to deal with the issue.

The wipe-dalvik function of a custom recovery wipes three folders: /data/dalvik-cache (internal /data), /cache/dalvik-cache (location used by CM roms), and /sd-ext/dalvik-cache (2nd partition of SD used by the hack for data). This zip wipes /sd-ext/dalvik-cache and /cache/dalvik-cache, but leaves /data/dalvik-cache alone. So this zip could be used to replace the wipe-dalvik function for users of my hack. As I said before, using the built-in wipe-dalvik also deletes the dalvik-cache folder itself. Without that folder, the hack wouldn't mount the partition. Flashing the zip when that folder is missing would recreate the missing folder, so the zip could be used after doing the wipe-dalvik function to fix the issue of the missing folder.
Note: for KK roms, this should to be flashed with a recovery that supports selinux. So on the F6, use TWRP.

Works Great.. Passed the test...

 

[aangbell]

I'M SUFFERING lag and I don't know why. In the beginning it was ok with sometimes a little lag but now it's most frequently. I have a 32 gb SanDisk MicroSD ultra plus and the last update of xperion Rom running in my device. Anyone could help me?

 

[WarrantyVoider]

@aangbell
Can you do a speed test on the card? If you have a Windows PC and a decent card reader, run CrystalDiskMark and let me know the read/write speed for the 4k test. Alternatively, install A1 SD Bench from the Play Store and do a Random I/O test on "Internal memory" with the hack active. Also, how did you partition the card and what did you use?

 

[aangbell]

I did the test days ago and the result it wasn't the same. In this test the results are low. I did the partition with partition wizard tools. First the fat32 partition then the Ext4 like you said in this guide.
I think that something it's wrong because the MicroSD it's no bad for this I suppose.

 

[WarrantyVoider]

Do the Random I/O test like I said. Long-press on "Internal memory" and choose "Random I/O." The test requires a reboot to finish. To do the test on the actual internal memory for comparison, flash -uninstall.zip in recovery. Run the test on "Internal memory" as before. To reactivate the hack again, flash -install.zip in recovery (skipping -copy.zip).

Did you purchase the card from Best Buy like in your link previously? If the card is genuine and the card performs noticeably worse now than when it was new, then the only thing I can think of right now is possibly misaligned partition(s) causing extra write/erase cycles. But the effect shouldn't have manifested already on a relatively new card.

 

[aangbell]

Do the Random I/O test like I said. Long-press on "Internal memory" and choose "Random I/O." The test requires a reboot to finish. To do the test on the actual internal memory for comparison, flash -uninstall.zip in recovery. Run the test on "Internal memory" as before. To reactivate the hack again, flash -install.zip in recovery (skipping -copy.zip).

Yes it's the same card. Ok i'm gonna do it.

 

[aangbell]

Here is the result of the test with the hack. Without the hack result less like 0.18 mb/s on write. Well the case is that without the hack result less speed on comparison with the hack applied. :/