瀏覽代碼

[sparksql] Clean unused session from /fetch_result_data call (#2867)

- /fetch_result_data is called only once after /execute call (unlike /check_status or /get_logs).
- We can leverage the call for cleaning up any unused sessions specially the dead ones which are created after cancelling the executed query statement and retrying.
Harsh Gupta 3 年之前
父節點
當前提交
fdf927d55e
共有 1 個文件被更改,包括 6 次插入0 次删除
  1. 6 0
      desktop/libs/notebook/src/notebook/connectors/spark_shell.py

+ 6 - 0
desktop/libs/notebook/src/notebook/connectors/spark_shell.py

@@ -234,6 +234,12 @@ class SparkApi(Api):
     session = self._handle_session_health_check(session)
 
     response = self._fetch_result(api, session, cell, start_over)
+
+    # Close unused sessions if there are any.
+    # Clean here since /fetch_result_data is called only once after the /execute call
+    if self._get_session_info_from_user():
+      self._close_unused_sessions()
+
     return response