Difference between revisions of "Tensorflow JNI development"

From ElphelWiki
Jump to: navigation, search
(Created page with "==About== Based on [https://medium.com/@maxime.durand.54/build-tennsorflow-2-0-for-java-on-windows-2ab51b9cac45 this]. How to: * Install things * Build libtensorflow.jar, li...")
 
(About)
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
==About==
 
==About==
Based on [https://medium.com/@maxime.durand.54/build-tennsorflow-2-0-for-java-on-windows-2ab51b9cac45 this].
 
  
How to:  
+
Based on [https://medium.com/@maxime.durand.54/build-tennsorflow-2-0-for-java-on-windows-2ab51b9cac45 Build TensorFlow 2.0 for Java on Windows] article.
* Install things
+
Also this one - [https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/README.md tensorflow/java/README.md].
* Build libtensorflow.jar, libtensorflow_jni.so
+
 
* Install pom.xml and libtensorflow.jar to local maven repo in ~/.m2
+
These instructions are for '''Linux''' and old '''TensorFlow 1.15.0'''.
* Modify Tensorflow JNI functions
+
 
* Create a maven project and use modified JNI
+
How to:
 +
* Build TF JNI - '''libtensorflow.jar, libtensorflow_jni.so'''
 +
* Add '''libtensorflow.jar''' to local Maven repository which will override the [https://mvnrepository.com/artifact/org.tensorflow/tensorflow Central Maven Repository]
 +
* Modify TF JNI functions
 +
* Create a Maven project and use modified TF JNI
 +
 
 +
There's [https://github.com/bytedeco/javacpp-presets JavaCPP Presets] project. But they fail to clearly explain what it does.
  
 
==Install==
 
==Install==
Line 16: Line 21:
 
==Build==
 
==Build==
 
  cd ~/git/tensorflow-1.15.0
 
  cd ~/git/tensorflow-1.15.0
  ./configure
+
  ./configure # do not forget CUDA
  bazel build -c opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni
+
  bazel build -c opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni //tensorflow/java:pom
bazel build -c opt //tensorflow/java:pom
+
 
mvn install:install-file -Dfile=bazel-bin/tensorflow/java/libtensorflow.jar -DpomFile=bazel-bin/tensorflow/java/pom.xml
+
With tensorflow bazel (when restarted?) tends to rebuild everything from scratch which takes a ton of time. Bazel starts its server which, to prevent it, add to ~/.bazelrc:
 +
startup --max_idle_secs=0
  
 
Artifacts of interest in bazel-bin/tensorflow/java/:
 
Artifacts of interest in bazel-bin/tensorflow/java/:
Line 28: Line 34:
 
* '''xml''' and '''jar''' are taken care of.
 
* '''xml''' and '''jar''' are taken care of.
 
* '''so''' will have to be in the library path - set LD_LIBRARY_PATH or PATH or go "java -Djava.library.path"
 
* '''so''' will have to be in the library path - set LD_LIBRARY_PATH or PATH or go "java -Djava.library.path"
 +
 +
==Install TF to local maven repo==
 +
mvn install:install-file -Dfile=bazel-bin/tensorflow/java/libtensorflow.jar -DpomFile=bazel-bin/tensorflow/java/pom.xml
 +
 +
How to uninstall maven local repo - and switch back to official versions form Central:
 +
https://stackoverflow.com/questions/15358851/how-to-remove-jar-file-from-local-maven-repository-which-was-added-with-install
  
 
==Modify TF JNI==
 
==Modify TF JNI==
Line 38: Line 50:
 
* add to header file tensorflow/java/src/main/native/tensorflow_jni.h
 
* add to header file tensorflow/java/src/main/native/tensorflow_jni.h
 
* add to c file tensorflow/java/src/main/native/tensorflow_jni.cc
 
* add to c file tensorflow/java/src/main/native/tensorflow_jni.cc
 +
 +
* rebuild and reinstall to mvn
 +
 +
==Java maven project in Eclipse==
 +
* Create a new maven project
 +
* Edit pom.xml:
 +
<project>
 +
  ...
 +
  <dependencies>
 +
    ...
 +
    <b><dependency>
 +
      <groupId>org.tensorflow</groupId>
 +
      <artifactId>libtensorflow</artifactId>
 +
      <version>1.15.0</version>
 +
    </dependency></b>
 +
    ...
 +
  </dependencies>
 +
  ...
 +
</project>
 +
 +
===Basic example code===
 +
tfhello.java:
 +
import org.tensorflow.TensorFlow;
 +
 +
public class tfhello{
 +
public static void main(String[] args){
 +
System.out.println(TensorFlow.version());
 +
}
 +
}

Revision as of 17:26, 4 March 2020

About

Based on Build TensorFlow 2.0 for Java on Windows article. Also this one - tensorflow/java/README.md.

These instructions are for Linux and old TensorFlow 1.15.0.

How to:

  • Build TF JNI - libtensorflow.jar, libtensorflow_jni.so
  • Add libtensorflow.jar to local Maven repository which will override the Central Maven Repository
  • Modify TF JNI functions
  • Create a Maven project and use modified TF JNI

There's JavaCPP Presets project. But they fail to clearly explain what it does.

Install

  • In Kubuntu
  • Get TF 1.15.0 - git or archive
  • Install bazel 0.25.2 - see *.deb in releases

Build

cd ~/git/tensorflow-1.15.0
./configure # do not forget CUDA
bazel build -c opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni //tensorflow/java:pom

With tensorflow bazel (when restarted?) tends to rebuild everything from scratch which takes a ton of time. Bazel starts its server which, to prevent it, add to ~/.bazelrc:

startup --max_idle_secs=0

Artifacts of interest in bazel-bin/tensorflow/java/:

libtensorflow_jni.so
libtensorflow.jar
pom.xml
  • xml and jar are taken care of.
  • so will have to be in the library path - set LD_LIBRARY_PATH or PATH or go "java -Djava.library.path"

Install TF to local maven repo

mvn install:install-file -Dfile=bazel-bin/tensorflow/java/libtensorflow.jar -DpomFile=bazel-bin/tensorflow/java/pom.xml

How to uninstall maven local repo - and switch back to official versions form Central:

https://stackoverflow.com/questions/15358851/how-to-remove-jar-file-from-local-maven-repository-which-was-added-with-install

Modify TF JNI

For example, one wants to create a new function in org.tensorflow.TensorFlow package. Then, see:

tensorflow/java/src/main/java/org/tensorflow/
tensorflow/java/src/main/native/
  • add native method to tensorflow/java/src/main/java/org/tensorflow/TensorFlow.java
  • add to header file tensorflow/java/src/main/native/tensorflow_jni.h
  • add to c file tensorflow/java/src/main/native/tensorflow_jni.cc
  • rebuild and reinstall to mvn

Java maven project in Eclipse

  • Create a new maven project
  • Edit pom.xml:
<project>
  ...
  <dependencies>
    ...
    <dependency>
      <groupId>org.tensorflow</groupId>
      <artifactId>libtensorflow</artifactId>
      <version>1.15.0</version>
    </dependency>
    ...
  </dependencies>
  ...
</project>

Basic example code

tfhello.java:

import org.tensorflow.TensorFlow;

public class tfhello{
	public static void main(String[] args){
		System.out.println(TensorFlow.version());
	}
}