|
@@ -0,0 +1,78 @@
|
|
|
|
|
+---
|
|
|
|
|
+title: Automatically checking documentation and website dead links with Continuous Integration
|
|
|
|
|
+author: Romain
|
|
|
|
|
+type: post
|
|
|
|
|
+date: 2020-03-11T00:00:00+00:00
|
|
|
|
|
+url: /checking-dead-links-automatically-continuous-integration/
|
|
|
|
|
+sf_thumbnail_type:
|
|
|
|
|
+ - none
|
|
|
|
|
+sf_thumbnail_link_type:
|
|
|
|
|
+ - link_to_post
|
|
|
|
|
+sf_detail_type:
|
|
|
|
|
+ - none
|
|
|
|
|
+sf_page_title:
|
|
|
|
|
+ - 1
|
|
|
|
|
+sf_page_title_style:
|
|
|
|
|
+ - standard
|
|
|
|
|
+sf_no_breadcrumbs:
|
|
|
|
|
+ - 1
|
|
|
|
|
+sf_page_title_bg:
|
|
|
|
|
+ - none
|
|
|
|
|
+sf_page_title_text_style:
|
|
|
|
|
+ - light
|
|
|
|
|
+sf_background_image_size:
|
|
|
|
|
+ - cover
|
|
|
|
|
+sf_social_sharing:
|
|
|
|
|
+ - 1
|
|
|
|
|
+sf_related_articles:
|
|
|
|
|
+ - 1
|
|
|
|
|
+sf_sidebar_config:
|
|
|
|
|
+ - left-sidebar
|
|
|
|
|
+sf_left_sidebar:
|
|
|
|
|
+ - Sidebar-2
|
|
|
|
|
+sf_right_sidebar:
|
|
|
|
|
+ - Sidebar-1
|
|
|
|
|
+sf_caption_position:
|
|
|
|
|
+ - caption-right
|
|
|
|
|
+ampforwp-amp-on-off:
|
|
|
|
|
+ - default
|
|
|
|
|
+categories:
|
|
|
|
|
+ - Administration
|
|
|
|
|
+ - Version 4.7
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+Hi Data Crunchers,
|
|
|
|
|
+
|
|
|
|
|
+Continuous integration and automation are investment that enable a major scaling in the resource and quality of software projects. This past year saw a lot of improvements with an [integrated commit flow](https://gethue.com/improving-the-developer-productivity-with-some-continuous-integration/) and adding series of checks like linting of JavaScript, also running Python 3 tests automatically...
|
|
|
|
|
+
|
|
|
|
|
+This also create a virtuous circle that encourages developers to add more tests on their own (e.g. +200 since the beginning of this year), as all the plumbing is already done for them.
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+CI workflow
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+List of CI checks
|
|
|
|
|
+
|
|
|
|
|
+Next item on the list to automate was the automated checking of deadlinks of https://docs.gethue.com and https://gethue.com/ that was previously [manual](https://gethue.com/easily-checking-for-deadlinks-on-docs-gethue-com/).
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+New website deadlinks check
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+Overall the [action script](https://github.com/cloudera/hue/blob/master/tools/ci/check_for_website_dead_links.sh) will:
|
|
|
|
|
+
|
|
|
|
|
+* Check if the new commits containing a documentation of website change
|
|
|
|
|
+* Then boot `hugo` to locally serve the site
|
|
|
|
|
+* And run `muffet` to crawl and checks the links
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+Link checking failure
|
|
|
|
|
+
|
|
|
|
|
+Note: it is handy to had some URL blacklist and lower number of concurrent crawler connections to avoid hammering some external websites (e.g. https://issues.cloudera.org/browse/HUE which contains a lot of references).
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+What is your favorite CI process? Any feedback? Feel free to comment here or on [@gethue](https://twitter.com/gethue)!
|