Thing Translator

Introduction

In Part I of this tutorial we built a Web API servicethat handles image labeling and translation into different languages. Now we will build Android app that can consume that API. Features we are going to have are:

Using the code

All the code base is located in code section of this article. For simplicity reasons here I am going to highlight only the important parts of the code.

build.gradle

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    compile 'com.google.code.gson:gson:2.6.1'
    compile 'com.android.support:support-v13:23.4.0'
    compile 'com.android.support:appcompat-v7:23.4.0'
    compile 'com.android.support:design:23.4.0'
    compile 'com.squareup.retrofit2:retrofit:2.0.0'
    compile 'com.squareup.retrofit2:converter-gson:2.0.0'
    compile 'com.squareup.okhttp3:okhttp:3.4.1'
    compile 'com.android.support.constraint:constraint-layout:1.0.0-alpha4'
    testCompile 'junit:junit:4.12'
    compile 'com.pixplicity.easyprefs:library:1.8.1@aar'
    compile 'com.jakewharton:butterknife:8.4.0'
    annotationProcessor 'com.jakewharton:butterknife-compiler:8.4.0'
}

We will use some really cool Java libraries that make Android development lot easier.

activity_main - spinner View has a custom layout defined in spinner_item.xml

        <Spinner
            android:id="@+id/spinner"
            android:layout_width="100dp"
            android:layout_height="match_parent"
            android:layout_marginBottom="16dp"
            android:layout_weight="0.91"
            android:gravity="bottom|right" />
 
<?xml version="1.0" encoding="utf-8"?>
<TextView
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="wrap_content"
    android:layout_height="30dp"
    android:textColor="@android:color/darker_gray"
    android:textSize="24dp"
    />

ApiInterface.java - this is an interface for our REST API calls. Put your API URL we build in the previous tutorial into ENDPOINT variable. As you can see only 2 parameters are used when making POST request:

  1. Image file
  2. Language Code
package thingtranslator2.jalle.com.thingtranslator2.Rest;
 
public interface ApiInterface {
    
    String ENDPOINT = "YOUR-API-URL/api/";
    @Multipart
    @POST("upload")
    Call<translation> upload(@Part("image\"; filename=\"pic.jpg\" ") RequestBody file, @Part("FirstName") RequestBody langCode1);
    final OkHttpClient okHttpClient = new OkHttpClient.Builder()
            .readTimeout(60, TimeUnit.SECONDS)
            .connectTimeout(60, TimeUnit.SECONDS)
            .build();
    Retrofit retrofit = new Retrofit.Builder()
            .baseUrl(ENDPOINT)
            .client(okHttpClient)
            .addConverterFactory(GsonConverterFactory.create())
            .build();
}

Translation class

package thingtranslator2.jalle.com.thingtranslator2.Rest;
public class Translation {
    public String Original;
    public String Translation;
    public String Error;
}

Tools - ImagePicker this class is used for getting an image from the gallery of taking a new photo from the camera.

Tools - MarshMallowPermission this is used for setting up camera permissions on Android 6.

AndroidManifest

    <?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="thingtranslator2.jalle.com.thingtranslator2">
    <uses-feature android:name="android.hardware.camera" />
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-feature android:name="android.hardware.camera.autofocus" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity" >
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>
</manifest>
    

MainActivity.java

    
public class MainActivity extends AppCompatActivity implements AdapterView.OnItemSelectedListener {
    public
    @BindView(R.id.txtTranslation)
    TextView txtTranslation;
    public
    @BindView(R.id.spinner)
    Spinner spinner;
    public
    @BindView(R.id.imgPhoto)
    ImageButton imgPhoto;
    public
    @BindView(R.id.imgSpeaker)
    ImageButton btnSpeaker;
    public Boolean speakerOn, firstRun;
    public TextToSpeech tts;
    public List<string> languages = new ArrayList<string>();
    public ArrayList<language> languageList = new ArrayList<language>();
    public ArrayAdapter<language> spinnerArrayAdapter;
    public Language selectedLanguage;
    public static final int PICK_IMAGE_ID = 234; // the number doesn't matter
    thingtranslator2.jalle.com.thingtranslator2.MarshMallowPermission marshMallowPermission = new thingtranslator2.jalle.com.thingtranslator2.MarshMallowPermission(this);
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN);
        ButterKnife.bind(this);
        //init our Shared preferences helper
        new Prefs.Builder()
                .setContext(this)
                .setMode(ContextWrapper.MODE_PRIVATE)
                .setPrefsName(getPackageName())
                .setUseDefaultSharedPreference(true)
                .build();
        //  Prefs.clear();
        firstRun = Prefs.getBoolean("firstRun", true);
        spinner.setOnItemSelectedListener(this);
        tts = new TextToSpeech(getApplicationContext(), new TextToSpeech.OnInitListener() {
            @Override
            public void onInit(int status) {
                if (status != TextToSpeech.ERROR) {
                    fillSpinner();
                }
            }
        });
        if (firstRun) {
            Prefs.putBoolean("firstRun", false);
            Prefs.putBoolean("speakerOn", true);
            Prefs.putString("langCode", "bs");
        }
        setSpeaker();
    }
    private void setSpeaker() {
        int id = Prefs.getBoolean("speakerOn", false) ? R.drawable.ic_volume : R.drawable.ic_volume_off;
        btnSpeaker.setImageBitmap(BitmapFactory.decodeResource(getResources(), id));
    }
    public void ToogleSpeaker(View v) {
        Prefs.putBoolean("speakerOn", !Prefs.getBoolean("speakerOn", false));
        setSpeaker();
        Toast.makeText(getApplicationContext(), "Speech " + (Prefs.getBoolean("speakerOn", true) ? "On" : "Off"), Toast.LENGTH_SHORT);
    }
    private void fillSpinner() {
        Iterator itr = tts.getAvailableLanguages().iterator();
        while (itr.hasNext()) {
            Locale item = (Locale) itr.next();
            languageList.add(new Language(item.getDisplayName(), item.getLanguage()));
        }
        //Sort that array
        Collections.sort(languageList, new Comparator<language>() {
            @Override
            public int compare(Language o1, Language o2) {
                return o1.getlangCode().compareTo(o2.getlangCode());
            }
        });
        spinnerArrayAdapter = new ArrayAdapter<language>(this, R.layout.spinner_item, languageList);
        spinnerArrayAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
        spinner.setAdapter(spinnerArrayAdapter);
        //Set the default language to bs- Bosnian
        String def = Prefs.getString("langCode", "bs");
        for (Language lang : languageList) {
            if (lang.getlangCode().equals(def)) {
                spinner.setSelection(languageList.indexOf(lang));
            }
        }
    }
    ;
    //Get image from  Gallery or Camera
    public void getImage(View v) {
        if (!marshMallowPermission.checkPermissionForCamera()) {
            marshMallowPermission.requestPermissionForCamera();
        } else {
            if (!marshMallowPermission.checkPermissionForExternalStorage()) {
                marshMallowPermission.requestPermissionForExternalStorage();
            } else {
                Intent chooseImageIntent = thingtranslator2.jalle.com.thingtranslator2.ImagePicker.getPickImageIntent(getApplicationContext());
                startActivityForResult(chooseImageIntent, PICK_IMAGE_ID);
            }
        }
    }
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (data == null) return;
        switch (requestCode) {
            case PICK_IMAGE_ID:
                Bitmap bitmap = thingtranslator2.jalle.com.thingtranslator2.ImagePicker.getImageFromResult(this, resultCode, data);
                imgPhoto.setImageBitmap(null);
                imgPhoto.setBackground(null);
                imgPhoto.setImageBitmap(bitmap);
                imgPhoto.invalidate();
                imgPhoto.postInvalidate();
                File file = null;
                try {
                    file = savebitmap(bitmap, "pic.jpeg");
                } catch (IOException e) {
                    e.printStackTrace();
                }
                uploadImage(file);
                break;
            default:
                super.onActivityResult(requestCode, resultCode, data);
                break;
        }
    }
    private void uploadImage(File file) {
        RequestBody body = RequestBody.create(MediaType.parse("image/*"), file);
        RequestBody langCode1 = RequestBody.create(MediaType.parse("text/plain"), selectedLanguage.langCode);
        final ProgressDialog progress = new ProgressDialog(this);
        progress.setMessage("Processing image...");
        progress.setProgressStyle(ProgressDialog.STYLE_SPINNER);
        progress.setIndeterminate(true);
        progress.show();
        ApiInterface mApiService = ApiInterface.retrofit.create(ApiInterface.class);
        Call<translation> mService = mApiService.upload(body, langCode1);
        mService.enqueue(new Callback<translation>() {
            @Override
            public void onResponse(Call<translation> call, Response<translation> response) {
                progress.hide();
                Translation result = response.body();
                txtTranslation.setText(result.Translation);
                txtTranslation.invalidate();
                if (Prefs.getBoolean("speakerOn", true)) {
                    tts.speak(result.Translation, TextToSpeech.QUEUE_FLUSH, null);
                }
            }
            @Override
            public void onFailure(Call<translation> call, Throwable t) {
                call.cancel();
                progress.hide();
                Toast.makeText(getApplicationContext(), "Error: " + t.getMessage(), Toast.LENGTH_LONG).show();
            }
        });
    }
    public static File savebitmap(Bitmap bmp, String fName) throws IOException {
        ByteArrayOutputStream bytes = new ByteArrayOutputStream();
        bmp.compress(Bitmap.CompressFormat.JPEG, 60, bytes);
        File f = new File(Environment.getExternalStorageDirectory()
                + File.separator + fName);
        f.createNewFile();
        FileOutputStream fo = new FileOutputStream(f);
        fo.write(bytes.toByteArray());
        fo.close();
        return f;
    }
    @Override
    public void onItemSelected(AdapterView parent, View view, int position, long id) {
        selectedLanguage = (Language) spinner.getSelectedItem();
        //  langCode = languages.get(position);
        tts.setLanguage(Locale.forLanguageTag(selectedLanguage.langCode));
        Prefs.putString("langCode", selectedLanguage.langCode);
    }
    @Override
    public void onNothingSelected(AdapterView parent) {
    }
}

The end result of this tutorial is an application that looks like this. Remember your WebAPI service needs to be up and running in order for all this to work.

Source code is available on GitHub

Points of Interest

There are many ways for improvement and upgrading this application. One of my ideas is that the image labeling could be done during the live preview on your camera. Having the preview window opened, we can invoke the API every 5-10 seconds and have live labeling about what we see through our camera.
One drawback is that it would consume a lot of bandwidth and also remember, Google API Vision and most of the other APIs are not free. You pay for the API usage when requests exceed the allowed quota.

Happy codding!