Add support for linking against a curses library when available and

using it to detect whether or not a terminal supports colors. This
replaces a particularly egregious hack that merely compared the TERM
environment variable to "dumb". That doesn't really translate to
a reasonable experience for users that have actually ensured their
terminal's capabilities are accurately reflected.

This makes testing a terminal for color support somewhat more expensive,
but it is called very rarely anyways. The important fast path when the
output is being piped somewhere is already in place.

The global lock may seem excessive, but the spec for calling into curses
is *terrible*. The whole library is terrible, and I spent quite a bit of
time looking for a better way of doing this before convincing myself
that this was the fundamentally correct way to behave. The damage of the
curses library is very narrowly confined, and we continue to use raw
escape codes for actually manipulating the colors which is a much sane
system than directly using curses here (IMO).

If this causes trouble for folks, please let me know. I've tested it on
Linux and will watch the bots carefully. I've also worked to account for
the variances of curses interfaces that I could finde documentation for,
but that may not have been sufficient.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187874 91177308-0d34-0410-b5e6-96231b3b80d8

1 files changed, 41 insertions, 10 deletions

diff --git a/lib/Support/Unix/Process.inc b/lib/Support/Unix/Process.inc
index 1335b78e86..0a797f6979 100644
--- a/lib/Support/Unix/Process.inc
+++ b/lib/Support/Unix/Process.inc
@@ -13,6 +13,8 @@
 
 #include "Unix.h"
 #include "llvm/ADT/Hashing.h"
+#include "llvm/Support/Mutex.h"
+#include "llvm/Support/MutexGuard.h"
 #include "llvm/Support/TimeValue.h"
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
@@ -36,6 +38,25 @@
 #  include <termios.h>
 #endif
 
+// See if we can use curses to detect information about a terminal when
+// connected to one.
+#ifdef HAVE_CURSES
+# if defined(HAVE_CURSES_H)
+#  include <curses.h>
+# elif defined(HAVE_NCURSES_H)
+#  include <ncurses.h>
+# elif defined(HAVE_NCURSESW_H)
+#  include <ncursesw.h>
+# elif defined(HAVE_NCURSES_CURSES_H)
+#  include <ncurses/curses.h>
+# elif defined(HAVE_NCURSESW_CURSES_H)
+#  include <ncursesw/curses.h>
+# else
+#  error Have a curses library but unable to find a curses header!
+# endif
+# include <term.h>
+#endif
+
 //===----------------------------------------------------------------------===//
 //=== WARNING: Implementation here must contain only generic UNIX code that
 //===          is guaranteed to work on *all* UNIX variants.
@@ -245,22 +266,32 @@ unsigned Process::StandardErrColumns() {
   return getColumns(2);
 }
 
-static bool terminalHasColors() {
-  if (const char *term = std::getenv("TERM")) {
-    // Most modern terminals support ANSI escape sequences for colors.
-    // We could check terminfo, or have a list of known terms that support
-    // colors, but that would be overkill.
-    // The user can always ask for no colors by setting TERM to dumb, or
-    // using a commandline flag.
-    return strcmp(term, "dumb") != 0;
-  }
+static bool terminalHasColors(int fd) {
+#ifdef HAVE_CURSES
+  // First, acquire a global lock because the curses C routines are thread
+  // hostile.
+  static sys::Mutex M;
+  MutexGuard G(M);
+
+  int errret = 0;
+  if (setupterm((char *)0, fd, &errret) != OK)
+    // Regardless of why, if we can't get terminfo, we shouldn't try to print
+    // colors.
+    return false;
+
+  // Test whether the terminal as set up supports color output.
+  if (has_colors() == TRUE)
+    return true;
+#endif
+
+  // Otherwise, be conservative.
   return false;
 }
 
 bool Process::FileDescriptorHasColors(int fd) {
   // A file descriptor has colors if it is displayed and the terminal has
   // colors.
-  return FileDescriptorIsDisplayed(fd) && terminalHasColors();
+  return FileDescriptorIsDisplayed(fd) && terminalHasColors(fd);
 }
 
 bool Process::StandardOutHasColors() {