![spark intellij jar spark intellij jar](https://docs.microsoft.com/en-us/azure/hdinsight/spark/media/apache-spark-create-standalone-application/hdinsight-create-jar3.png)
The latter method is easier than it sounds: you can intercept traffic and see requests and responses.
#Spark intellij jar code
We can go from the server side (the open-source code for this is available on GitHub), or from the client side (a closed-source part of IntelliJ IDEA). Let’s try to understand how Zeppelin finds a list of local variables and columns. At the time of the paragraph’s execution, it analyzes the context on the server and sends it to the client. It looks and feels like magic, and this is what ZTools provides us. And if that’s not enough, you can now autocomplete columns of the sql.DataFrame in the editor. This simple feature is an alternative to the debugger and it’ll save you a lot of time. Wait, what? How is it possible to view the variables after the execution? Now you can observe the values of local variables: When you add a new notebook to Big Data Tools, there’s an option to enable ZTools integration. Nevertheless, this example shows how it might look like in your own product. This plugin is a part of IntelliJ Ultimate Edition, and of course, its code is not open. To understand what ZTools can do, we’ll look at the Variables View panel and code editor in the Big Data Tools plugin. ZTools will actually work with any Scala REPL, so it is not just for Zeppelin, but for dealing with Scala in general. About 90% of the code is written in Scala, and the rest is in Java.
#Spark intellij jar license
The server-side code is publicly available on GitHub under the Apache License 2.0. It’s a tool that allows you to pull information from Zeppelin, even if it’s not present in the API. You need much lower-level access to do things like smart autocompletion or visual debugging. The high-level API for creating and removing notebooks is not enough for the IDE.
![spark intellij jar spark intellij jar](https://i.ytimg.com/vi/U7eNRAFwp0Y/maxresdefault.jpg)
That was a real problem for Big Data Tools, a plugin for IntelliJ IDEA that provides integration with Spark, Hadoop, and makes it possible to edit and run notebooks in Zeppelin. There is already an API that completely solves the problems of high-level notebook management, but it doesn’t help if you want to do anything more complex. One such thing is an API for getting comprehensive information about what’s going on inside the notebook. The project recently reached version 0.9.0-preview2 and is being actively developed, but there are still many things to be implemented. Zeppelin is a web-based notebook for data engineers that enables data-driven, interactive data analytics with Spark, Scala, and more.