ArthurGray
Newbie
Hi folks, I really need your help, having a hard time trying to debug this..
I'm building a custom video streaming service. I have a server application written in Python which is able to send a stream to my Android app. To display the contents, I'm using the native MediaPlayer class, and I've written my own media source class (called VideoDataSource) for it by extending MediaDataSource. I'll add the source code to the end of this post, but I think it's rather tedious so first I'll explain it so that you may just get some intuition to what is it I'm doing wrong here.
Operation --
The operating principle is simple: in the main activity creates an instance of VideoDataSource. within its constructor, this instance starts a new thread which uses the socket library to communicate with my server. When a new chunk of stream data is received by this thread, it proceeds to lock (or wait for unlocking) of a local file and flushes there all the data.
In the meantime, my main activity calls prepareAsync method to prepare the mediaplayer. This uses my VideoDataSource methods getSize and readAt. I have getSize always return -1 (size unknown), while readAt sleeps until the local file is unlocked, then proceeds to lock it and check if the data requested by the mediaplayer is ready. If so, it dumps it to the buffer provided by readAt call. If not, it keeps sleeping untill the other thread has downloaded it.
I have all of this run in the emulator with an mp4 video sample and it seems to be working fine. Whenever I try the same on my smartphone, the MediaPlayer throws the preparation error exception, as if the data is wrong and it doesen't recognize the type of data.
-- Debug attempts --
If I go and look at the downloaded files with the emulator and the smartphone, they are exactly the same.
I started suspecting that it was a synchronization problem of some kind, since the emulator was in the same machine as the server it received the packets immediately, so I tried to delay the prepareAsync call by 10 seconds as to allow the smartphone to gather all info, with no success. Then I tried to log manually all the calls to readAt - I logged the input parameter values, the returned value (number of bytes read) as well as the raw data retrieved - to text files with both emulator and smartphone, then compare them one by one. Turns out the first 74 calls to readAt are exactly the same. Same input args, same output value, same data. The 75 is different in that in the smartphone requests a higher size of data. the following calls also are different, up to the 112th, when the smartphone just gives up and throws the exception. Now i thought that maybe my smartphone had a different version of the MediaPlayer for some reason, even if they both have API version 29 (my smartphone is a oneplus 6 which just recently received Pie update). Finally, I tried to manually crop the sample video file in the server to a few KB, so that it could be sent entirely with a single tcp packet... and the smartphone now visualizes it correctly. To this, i don't know exactly what to think. When the smarpthone failed, it was reading only data from the first packet either way, thus the reception of a second packet does not affect the functioning of this. I even tried back with the normal length sequence and a modified version of the server which only sends the first packet. And the video wasn't displayed. The app has no way to know how big is the source file..this is driving me crazy.
Now for the code, I'll post the major classes. If you need more, i'll upload the whole project..
MainActivity.java
VideoDataSource.java
receiveThread.java
Thanks in advance to anyone who'll have the patience to help me figure out what's wrong
I'm building a custom video streaming service. I have a server application written in Python which is able to send a stream to my Android app. To display the contents, I'm using the native MediaPlayer class, and I've written my own media source class (called VideoDataSource) for it by extending MediaDataSource. I'll add the source code to the end of this post, but I think it's rather tedious so first I'll explain it so that you may just get some intuition to what is it I'm doing wrong here.
Operation --
The operating principle is simple: in the main activity creates an instance of VideoDataSource. within its constructor, this instance starts a new thread which uses the socket library to communicate with my server. When a new chunk of stream data is received by this thread, it proceeds to lock (or wait for unlocking) of a local file and flushes there all the data.
In the meantime, my main activity calls prepareAsync method to prepare the mediaplayer. This uses my VideoDataSource methods getSize and readAt. I have getSize always return -1 (size unknown), while readAt sleeps until the local file is unlocked, then proceeds to lock it and check if the data requested by the mediaplayer is ready. If so, it dumps it to the buffer provided by readAt call. If not, it keeps sleeping untill the other thread has downloaded it.
I have all of this run in the emulator with an mp4 video sample and it seems to be working fine. Whenever I try the same on my smartphone, the MediaPlayer throws the preparation error exception, as if the data is wrong and it doesen't recognize the type of data.
-- Debug attempts --
If I go and look at the downloaded files with the emulator and the smartphone, they are exactly the same.
I started suspecting that it was a synchronization problem of some kind, since the emulator was in the same machine as the server it received the packets immediately, so I tried to delay the prepareAsync call by 10 seconds as to allow the smartphone to gather all info, with no success. Then I tried to log manually all the calls to readAt - I logged the input parameter values, the returned value (number of bytes read) as well as the raw data retrieved - to text files with both emulator and smartphone, then compare them one by one. Turns out the first 74 calls to readAt are exactly the same. Same input args, same output value, same data. The 75 is different in that in the smartphone requests a higher size of data. the following calls also are different, up to the 112th, when the smartphone just gives up and throws the exception. Now i thought that maybe my smartphone had a different version of the MediaPlayer for some reason, even if they both have API version 29 (my smartphone is a oneplus 6 which just recently received Pie update). Finally, I tried to manually crop the sample video file in the server to a few KB, so that it could be sent entirely with a single tcp packet... and the smartphone now visualizes it correctly. To this, i don't know exactly what to think. When the smarpthone failed, it was reading only data from the first packet either way, thus the reception of a second packet does not affect the functioning of this. I even tried back with the normal length sequence and a modified version of the server which only sends the first packet. And the video wasn't displayed. The app has no way to know how big is the source file..this is driving me crazy.
Now for the code, I'll post the major classes. If you need more, i'll upload the whole project..
MainActivity.java
Java:
package com.example.andre.videoviewer2;
import android.app.Activity;
import android.os.Bundle;
import android.media.MediaPlayer;
import android.util.Log;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import com.example.andre.videoviewer2.StreamServices.VideoDataSource;
public class MainActivity extends Activity
implements SurfaceHolder.Callback, MediaPlayer.OnPreparedListener
{
private MediaPlayer mp = null;
private SurfaceHolder surfaceHolder = null;
private SurfaceView surfaceView = null;
@Override
protected void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
surfaceView = (SurfaceView)findViewById(R.id.surfaceView);
surfaceHolder = surfaceView.getHolder();
surfaceHolder.addCallback(MainActivity.this);
}
@Override
public void surfaceCreated(SurfaceHolder holder)
{
mp = new MediaPlayer();
mp.setDisplay(surfaceHolder);
VideoDataSource vds = new VideoDataSource(18892, this);
mp.setDataSource(vds);
mp.setOnPreparedListener(this);
try {
Thread.sleep(10000);
mp.prepareAsync();
}
catch(Exception e)
{
Log.e("ANTANI",e.getMessage()+"\nMainActivity.java:"+e.getStackTrace()[0].getLineNumber());
}
}
@Override
protected void onResume() {
super.onResume();
}
@Override
protected void onPause() {
super.onPause();
releaseMediaPlayer();
}
@Override
protected void onDestroy() {
super.onDestroy();
releaseMediaPlayer();
}
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
}
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
}
@Override
public void onPrepared(MediaPlayer mp) {
Log.d("ANTANI","onPrepared received");
this.mp.start();
}
private void releaseMediaPlayer()
{
if( mp != null )
{
mp.release();
mp = null;
}
}
}
VideoDataSource.java
Java:
package com.example.andre.videoviewer2.StreamServices;
import android.content.Context;
import android.media.MediaDataSource;
import android.os.Environment;
import android.util.Log;
import com.example.andre.videoviewer2.Const;
import com.example.andre.videoviewer2.Utils;
import java.io.File;
import java.io.FileOutputStream;
import java.io.OutputStreamWriter;
import java.io.RandomAccessFile;
import java.net.Socket;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
public class VideoDataSource extends MediaDataSource
{
public dataBox db = new dataBox();
private Thread t = null;
private List getChunkChain( int position, int size ) {
// position and size describe a chunk of data from the original source file.
// If the chunk of data has been entirely downloaded, this method returns a list
// of ChunkInfo objects describing the locations inside the resource file where to
// look for the data. If the chunk has not been downloaded, returns null
int count = db.chunkTable.size();
if( count == 0 )
return null;
boolean completed = false;
List chunkChain = new ArrayList();
ChunkInfo c;
int bytesToRead = size;
while( !completed ) {
boolean added = false;
for (int i = 0; i < count; i++) {
c = ((ChunkInfo) db.chunkTable.get(i));
if (position >= c.position && position <= c.position+c.length-1) {
chunkChain.add(c);
int availableBytes = c.position+c.length-position;
if( availableBytes >= bytesToRead )
completed = true;
bytesToRead -= availableBytes;
added = true;
break;
}
}
if (! added ) return null;
}
return chunkChain;
}
private int chunkChainCount = 0;
public int readAt(long position, byte[] buffer, int offset, int size) {
// Read size bytes from source at position pos inside buffer at position offset
try {
int pos = (int) position;
Log.d("ANTANI-readAt",String.format(Locale.US,"call readAt(offs:%d,size%d)",pos,size));
// wait for size packet reception
int sleepCount = 0;
while(db.waitingForVideoSize) {
Thread.sleep(50 );
if(sleepCount == 0) {
Log.d("ANTANI-readAt","waiting for video size");
sleepCount = 1;
}
}
// no data
if(db.videoSize == 0) return(0);
// end of stream
if(position >= db.videoSize) return (-1);
// Wait untill data is completely received
List chunkChain = getChunkChain(pos, size);
sleepCount = 0;
while (chunkChain == null){
Thread.sleep(30);
chunkChain = getChunkChain(pos, size);
if(sleepCount == 0) {
Log.d("ANTANI-readAt","waiting for data to be ready");
sleepCount = 1;
}
//Log.d("ANTANI","readAt: waiting for data to be available");
}
chunkChainCount += 1;
// Wait while file is locked
sleepCount = 0;
while( db.fileLocked ) {
Thread.sleep(20);
if( sleepCount == 0) {
sleepCount = 1;
Log.d("ANTANI-readAt","waiting for file to be free");
}
}
// Lock file and follow the chain to read all the data into buffer
db.fileLocked = true;
int bytesToRead, readBytes = 0, availableBytes;
RandomAccessFile raf = new RandomAccessFile(db.trackResource,"r");
for(int i=0;i<chunkChain.size();i++) {
ChunkInfo c = (ChunkInfo)chunkChain.get(i);
if(i == 0) {
raf.seek(c.offset + pos - c.position);
availableBytes = c.length - (pos - c.position);
bytesToRead = availableBytes < size ? availableBytes : size;
}
else {
raf.seek(c.offset);
bytesToRead = size-readBytes < c.length ? size-readBytes : c.length;
}
raf.read(buffer,offset+readBytes, bytesToRead);
readBytes += bytesToRead;
}
/*if( chunkChainCount == 73 ) {
String msg = "";
File logg = new File(
db.context.getExternalFilesDir(Environment.DIRECTORY_MOVIES),
"bad"+Integer.toString(chunkChainCount)+".txt"
);
FileOutputStream out = new FileOutputStream(logg);
int doneBytes = 0, doAmount, pass = 0;
while (true) {
pass += 1;
msg = "fulldata" + Integer.toString(pass) + ":[";
doAmount = 100 < size - doneBytes ? 100 : size - doneBytes;
if (doAmount == 0) break;
for (int jj = 0; jj < doAmount; jj++)
msg += Byte.toString(buffer[offset + doneBytes + jj]) + ",";
doneBytes += doAmount;
msg += "]\n";
out.write(msg.getBytes());
}
out.close();
}*/
// free the file and return
raf.close();
db.fileLocked = false;
Log.d("ANTANI-readAt","read "+Integer.toString(readBytes)+" bytes.");
return readBytes;
}
catch(Exception ioe)
{
Log.e("ANTANI",ioe.getMessage());
return 0;
}
}
public long getSize() {
return -1;
}
public void close() {
dispose();
}
public VideoDataSource(int trackID, Context context) {
dispose();
try {
db.trackID = trackID;
db.context = context;
// Delete track resource file if present, create file resource for later access
File ff = new File(context.getExternalFilesDir(Environment.DIRECTORY_MOVIES),
Integer.toString(db.trackID)+".res");
if(ff.exists())
if(!ff.delete())
Log.e("ANTANI","unable to delete resource file!");
db.trackResource = ff;
// Start new thread to receive the stream
t = new Thread( new receiveThread(db));
t.start();
}
catch( Exception e ) {
Log.e("ANTANI",e.getMessage()+"\nVideoDataSource.java:"+e.getStackTrace()[0].getLineNumber());
}
}
private void dispose() {
try {
if (db.sock != null)
{
Log.d("ANTANI","called dispose..");
//Utils.sendPacket(db.sock,new byte[] {(byte)Const.PACKET_QUIT},0,1);
//db.sock.close();
//db.sock = null;
}
if (t != null)
{
//t.interrupt();
//t = null;
}
}
catch( Exception e )
{
Log.e("ANTANI",e.getMessage()+"\nVideoDataSource.java:"+e.getStackTrace()[0].getLineNumber());
}
}
}
receiveThread.java
Java:
package com.example.andre.videoviewer2.StreamServices;
import android.util.Log;
import com.example.andre.videoviewer2.Const;
import com.example.andre.videoviewer2.Utils;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.net.Socket;
import java.nio.ByteBuffer;
class receiveThread implements Runnable {
private Socket sock = null;
private InputStream inStream = null;
private byte[] inBuff = new byte[1024];
private byte[] packetBuffer = new byte[8000000];
private dataBox db;
receiveThread(dataBox db) {
this.db = db;
}
private int globalFileOffset;
public void run() {
try {
if(sock == null) {
globalFileOffset = 0;
String host = "192.168.1.70"; //"10.0.2.2";
sock = new Socket(host, 9934);
db.sock = sock;
inStream = db.sock.getInputStream();
// Send the first packet to the server requesting the desired track
byte[] intArr = ByteBuffer.allocate(4).putInt(db.trackID).array();
Utils.sendPacket(db.sock, new byte[]{(byte) Const.PACKET_REQUEST, intArr[0], intArr[1], intArr[2], intArr[3]}, 0, 5);
}
int bytesRead = 0;
int offset = 0;
while ((bytesRead = inStream.read(inBuff,0, inBuff.length)) != -1)
{
// Collect phase: collect the received bytes inside packetBuffer until the
System.arraycopy(inBuff,0,packetBuffer, offset, bytesRead);
offset += bytesRead;
int dataLen, surplus;
if(offset > 1) {
switch ((int) packetBuffer[0]) {
case Const.PACKET_CHUNK:
if (offset < 9) break;
dataLen = ByteBuffer.wrap(packetBuffer, 5, 4).getInt();
if(offset >= dataLen + 9) {
// packet complete, do what you need
int pos = ByteBuffer.wrap(packetBuffer, 1, 4).getInt();
// Wait for file to be writable
while (db.fileLocked) Thread.sleep(1);
// Lock the file for my write operation
db.fileLocked = true;
FileOutputStream out = new FileOutputStream(db.trackResource, true);
out.write(packetBuffer, 9, dataLen);
out.close();
db.chunkTable.add(new ChunkInfo(pos, dataLen, globalFileOffset));
globalFileOffset += dataLen;
db.fileLocked = false;
surplus = offset - (9+dataLen);
for(int i=0; i<surplus; i++)
packetBuffer[i] = packetBuffer[9+dataLen+i];
offset = surplus;
Log.d("ANTANI","received chunk of data!");
Utils.sendPacket(db.sock,new byte[]{(byte)Const.PACKET_ACK},0,1);
break;
}
break;
// note: db.videoSize and db.waitingForVideoSize are not actually used
case Const.PACKET_SIZE:
if( offset < 5 ) break;
// packet complete: do what you need to do
db.videoSize = ByteBuffer.wrap(packetBuffer,1,4).getInt();
db.waitingForVideoSize = false;
// reset offset
Log.d("ANTANI","received size");
surplus = offset - (5);
for (int i = 0; i < surplus; i++)
packetBuffer[i] = packetBuffer[5 + i];
offset = surplus;
break;
default:
Log.e("ANTANI","unknown type packet received");
}
}
}
}
catch( Exception e ) {
Log.e("ANTANI",e.getMessage()+"\nreceiveThread.java:"+e.getStackTrace()[0].getLineNumber());
}
}
}
Last edited: