Issue when clicking host: failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error

So this broke after another person worked on a more advanced aspect of what were trying to do. Forwarding graphite to a remote host and now we can’t get graphing data to show (the guy tried something complicated, he pointed graphite to a remote grafana host and then pointed the whisper/graphite back to the icinga i think)

Link of the error.

I’ll attach as much config info as I can think of.

[graphite]
metric_prefix = icinga
; base_url = http://localhost/render?
base_url = http://grafana.xxx.xxx:8080/render?

; fetch remote image and inline the data
remote_fetch = true
; verify remote certificate
; remote_verify_peer = true
remote_verify_peer = false
; verify remote peer name
; remote_verify_peer_name = true
remote_verify_peer_name = false

legacy_mode = false

; if legacy mode is false (2.4 and newer):
service_name_template = "icinga2.$host.name$.services.$service.name$.$service.check_command$.perfdata.$metric$.value"
host_name_template = "icinga2.$host.name$.host.$host.check_command$.perfdata.$metric$.value"

; if legacy mode is true (pre 2.4):
;service_name_template = "icinga.$host.name$.services.$service.name$.$service.check_command$"
;host_name_template = "icinga.$host.name$.host.$host.check_command$"
;this template is used for the small image, macros $target$ , $areaMode$ and $areaAlpha$ can used.
graphite_args_template = "&target=$target$&source=0&width=300&height=120&hideAxes=true&lineWidth=2&hideLegend=true&colorList=$colorList$&areaMode=$areaMode$&areaAlpha=$areaAlpha$"
;this template is used for the large image, macros $target$ , $areaMode$ and $areaAlpha$ can used.
;graphite_large_args_template = "&target=alias(color($target$_warn,'yellow'),'warning')&target=alias(color($target$_crit,'red'),'critical')&target=$target$&source=0&width=800&height=700&colorList=$colorList$&lineMode=connected&areaMode=$areaMode$&areaAlpha=$areaAlpha$"
graphite_iframe_w = 800px
graphite_iframe_h = 700px
graphite_area_mode = all
graphite_area_alpha = 0.1
graphite_summarize_interval = 10min
graphite_color_list = 049BAF,EE1D00,04B06E,0446B0,871E10,CB315D,B06904,B0049C

/etc/carbon/carbon.conf

# This is a list of carbon daemons we will send any relayed or
# generated metrics to. The default provided would send to a single
# carbon-cache instance on the default port. However if you
# use multiple carbon-cache instances then it would look like this:
#
# DESTINATIONS = 127.0.0.1:2004:a, 127.0.0.1:2104:b
#
# The general form is IP:PORT:INSTANCE where the :INSTANCE part is
# optional and refers to the "None" instance if omitted.
#
# Note that if the destinations are all carbon-caches then this should
# exactly match the webapp's CARBONLINK_HOSTS setting in terms of
# instances listed (order matters!).
#
# If using RELAY_METHOD = rules, all destinations used in relay-rules.conf
# must be defined in this list
DESTINATIONS = 127.0.0.1:2004, 192.168.5.5:2003
/**
 * The GraphiteWriter type writes check result metrics and
 * performance data to a graphite tcp socket.
 */

library "perfdata"

object GraphiteWriter "graphite" {
  host = "192.168.32.28"
  port = 2003
  enable_send_thresholds = true
  enable_send_metadata = true
}

All I can think of right now

At this point I’m leaning towards a fresh reinstall because this is pretty broke now. Maybe someone can save it.

500, or internal server error, should always lead you into the application and its log. In this case graphite-web itself. Depending on your installation method, they’re put into /opt/graphite or likewise /var/log.