[ZEPPELIN-6430] Remove SparkR and R interpreter#5271
Open
pan3793 wants to merge 15 commits into
Open
Conversation
The selenium integration test job (test-selenium-with-spark-module-for-spark-3-5) fails on AuthenticationIT because the GitHub runner ships Chrome 149 while the pinned Selenium 4.20.0 only bundles devtools up to CDP v125, producing "Unable to find CDP implementation matching 149" and degrading browser automation. This failure is pre-existing on master and unrelated to the SparkR/R removal in this PR. Selenium 4.45.0 bundles selenium-devtools-v147/v148/v149, an exact match for Chrome 149. The integration tests only use stable Selenium 4.x APIs, so the bump is API-compatible.
Removing R packages from the conda env files changed the dependency solve,
so conda started pulling protobuf >= 4.x/5.x. Zeppelin ships an old-format
generated kernel_pb2.py (interpreter/jupyter/grpc/jupyter/kernel_pb2.py),
which is incompatible with protobuf 4+, causing IPythonKernelTest to fail:
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date
... Downgrade the protobuf package to 3.20.x or lower.
core-modules and interpreter-test-jupyter-python both hit this (master passed
because the R-bearing env kept protobuf at 3.x). Pin protobuf<4, matching the
existing convention in env_python_3_with_flink_{119,120}.yml.
- scripts/vagrant/zeppelin-dev/README.md: drop the R/R-packages bullet (the R vagrant role was already removed). - docs/interpreter/jupyter.md: remove the "Jupyter R kernel" (IRKernel) section, consistent with dropping IRKernel support/tests. - message-common.interface.ts: drop 'r' / 'ace/mode/r' from the EditorLanguage/EditorMode unions (no interpreter emits them anymore).
No longer referenced after dropping the "Jupyter R kernel" section in docs/interpreter/jupyter.md.
Member
|
@pan3793 Thank you for the work. FYI, we could remove some licenses. IIRC, we added some license when adopting R relevant changes. |
The Shiro FormAuthenticationFilter (aliased 'authc') intercepts POSTs to
/api/login because the [urls] section lacked /api/login = anon and the
catch-all /** = authc matches it. The filter reads request.getParameter
("username") — consuming the request body — before Jersey's @FormParam
can parse "userName", resulting in a null username and
UnknownAccountException. Adding /api/login = anon bypasses the filter,
letting LoginRestApi.postLogin handle authentication directly.
Fixes AuthenticationIT, PersonalizeActionsIT, InterpreterModeActionsIT
which all use shiro.loginUrl = /api/login.
…x overlap The classic-webapp's fixed navbar logo occasionally overlaps the login modal buttons, causing ElementClickInterceptedException. Added a JavascriptExecutor-based click fallback to both the navbar Login button and the modal Login button in authenticationUser(), bypassing the purely visual z-index/stacking issue.
Member
Author
|
@jongyoul I've performed a check. The LICENSE, NOTICE, and |
Member
|
@pan3793 Thank you for confirming it! |
The login modal buttons can throw ElementNotInteractable (e.g. while the Bootstrap modal fade animation is still settling) in addition to ElementClickIntercepted (navbar logo z-index overlap). Extend both catch blocks in authenticationUser to fall back to the JavascriptExecutor click for either exception.
…nteractable ElementClickInterceptedException extends ElementNotInteractableException in Selenium 4.x, making Java multi-catch | illegal for related types. Catch only the supertype ElementNotInteractableException which covers both.
sendKeys dispatches DOM input events that only update Angular's $viewValue. The $modelValue (loginParams.userName) is only committed on the next $digest, but ng-click's $apply(fn) runs fn() before the digest, so login() reads stale (empty) credentials. The AJAX POST silently sends userName= or never dispatches. Replace sendKeys with direct Angular scope injection: find the scope containing loginParams, set userName and password directly, then call $apply() to digest. This guarantees the model is correct when ng-click fires login().
The scope-injection approach was unnecessary complexity. Instead, keep the
original sendKeys flow but add an angular.element('#loginModal').scope().$apply()
after typing to commit pending ng-model $viewValue changes before the login
button's ng-click handler runs $apply(fn) — which calls the login function
before the digest, causing it to read stale (empty) model values.
Chrome 149 changes the order Angular adds classes. The xpath contains(@Class, 'nav-btn dropdown-toggle ng-scope') fails when Angular prepends ng-scope (producing 'ng-scope nav-btn dropdown-toggle') instead of appending. Remove ng-scope from the contains() substring — nav-btn dropdown-toggle is unique enough in the navbar.
The AuthenticationIT login-modal flakiness under Chrome 149 is a pre-existing issue on master, unrelated to the SparkR/R removal. All integration-test CI fixes (Selenium bump, Shiro /api/login=anon, authenticationUser robustness) are reverted here. The other fixes (protobuf pins, R doc/image cleanup, conda env cleanup) are retained.
Member
Author
|
@jongyoul, it's ready now. The two frontend workflows fail consistently and are unrelated to this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this PR for?
SparkR has been officially deprecated since Spark 4.0, and plans to be removed in Spark 5.0 (early 2027), it also lacks maintenance and user cases in Zeppelin.
https://lists.apache.org/thread/qjgsgxklvpvyvbzsx1qr8o533j4zjlm5
What type of PR is it?
Breaking change.
What is the Jira issue?
ZEPPELIN-6430
How should this be tested?
Pass GHA
Screenshots (if appropriate)
Questions:
Assisted-by: GLM 5.2