Commit 21f0e36a authored by Grant Paton-Simpson's avatar Grant Paton-Simpson

Massively improve CLI viewer line wrapping; misc other changes

* Massively improve line wrapping in CLI viewer including monkey patch of textwraps _wrap_chunks
* Misc style changes in CLI viewer
* Use image linkes in README that PyPI can use
* Name transmission control characters STX and ETX
* Bump version
* Clarify use of HTML placeholders in CLI code
* Fix minor spelling and styling mistakes
* Remove unused function
parent bcf0dc7c
Pipeline #564 failed with stages
......@@ -13,4 +13,4 @@ upload:
rm -f dist/*
/home/g/projects/superhelp/superhelp/env/bin/python3 setup.py sdist bdist_wheel
/home/g/projects/superhelp/superhelp/env/bin/python3 -m twine upload dist/*
/home/g/projects/superhelp/superhelp/env/bin/python3 -m twine upload dist/* example_*.png
# https://git.nzoss.org.nz/pyGrant/superhelp
version number: 0.9.3
version number: 0.9.4
author: Grant Paton-Simpson
## Overview
Superhelp is Help for Humans! The goal is to provide customised help for simple
code snippets. Superhelp is not intended to replace the built-in Python help but
to supplement it for basic Python code structures. Superhelp will also be
opinionated. Help can be provided in a variety of contexts including the
terminal and web browsers (perhaps as part of on-line tutorials).
Superhelp is Help for Humans! The goal is to provide customised help for
simple code snippets. Superhelp is not intended to replace the built-in Python
help but to supplement it for basic Python code structures. Superhelp will
also be opinionated. Help can be provided in a variety of contexts including
the terminal and web browsers (perhaps as part of on-line tutorials).
## Quick Start
Click the button below to open a Binder Jupyter Notebook you can play around in
e.g. get advice on a line or snippet of Python
Click the button below to open a Binder Jupyter Notebook you can play around
in e.g. get advice on a line or snippet of Python
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/git/https%3A%2F%2Fgit.nzoss.org.nz%2FpyGrant%2Fsuperhelp.git/master?filepath=notebooks%2FSuperhelpDemo.ipynb)
......@@ -40,12 +40,12 @@ or similar
## Example Use Cases
* Charlotte is a Python beginner and wants to get advice on a five-line function
she wrote to display greetings to a list of people. She learns about Python
conventions for variable naming and better ways of combining strings.
* Charlotte is a Python beginner and wants to get advice on a five-line
function she wrote to display greetings to a list of people. She learns about
Python conventions for variable naming and better ways of combining strings.
* Avi wants to get advice on a named tuple. He learns how to add doc strings to
individual fields.
* Avi wants to get advice on a named tuple. He learns how to add doc strings
to individual fields.
* Zach is considering submitting some code to Stack Overflow but wants to
improve it first (or possibly get ideas for a solution directly). He discovers
......@@ -57,18 +57,18 @@ anything which can be improved. She learns how to use functool.wrap from an
example provided.
* Al is an experienced Python developer but tends to forget things like doc
strings in his functions. He learns a standard approach and starts using it more
often.
strings in his functions. He learns a standard approach and starts using it
more often.
# Example Usage
## Screenshot from HTML
![Example HTML output](example_html_output_1.png)
![Example HTML output](https://git.nzoss.org.nz/pyGrant/superhelp/-/raw/master/example_html_output_1.png)
## Screenshot from Terminal
![Example Terminal output](example_terminal_output_1.png)
![Example Terminal output](https://git.nzoss.org.nz/pyGrant/superhelp/-/raw/master/example_terminal_output_1.png)
## Notebook
......
example_terminal_output_1.png

122 KB | W: | H:

example_terminal_output_1.png

98 KB | W: | H:

example_terminal_output_1.png
example_terminal_output_1.png
example_terminal_output_1.png
example_terminal_output_1.png
  • 2-up
  • Swipe
  • Onion skin
......@@ -2,7 +2,7 @@ from setuptools import setup, find_packages # @UnresolvedImport
from codecs import open
from os import path
__version__ = '0.9.3'
__version__ = '0.9.4'
here = path.abspath(path.dirname(__file__))
......
......@@ -46,6 +46,7 @@ def exception_overview(blocks_dets):
block_comment_bits.append(layout(f"""\
#### `try`-`except` block{counter}
The following exception handlers were detected: {handlers}
"""))
block_comments = ''.join(block_comment_bits)
......
......@@ -219,6 +219,7 @@ def func_overview(block_dets, *, repeated_message=False):
details = ''.join(detail_bits)
if not repeated_message:
args_vs_params = layout(f"""\
There is often confusion about the difference between arguments and
parameters. {overall_func_type_lbl.title()}s define parameters but
receive arguments. You can think of parameters as being like car
......@@ -539,6 +540,7 @@ def docstring_issues(block_dets, *, repeated_message=False):
if first and not repeated_message: ## only want to say it once ;-)
summary_bits.append((
layout(f"""\
#### {func_type_lbl.title()} missing doc string
`{func_name}` lacks a doc string - you should probably add one.
......@@ -546,9 +548,9 @@ def docstring_issues(block_dets, *, repeated_message=False):
Note - # comments at the top of the {func_type_lbl} do
not work as doc strings. Python completely ignores them.
If you add a proper doc string, however, it can be
accessed by running help({func_name}) or
{func_name}.\_\_doc\_\_. Which is useful when using this
{func_type_lbl} in bigger projects e.g. in an IDE
accessed by running `help({func_name})` or
`{func_name}.`\_\_doc\_\_. Which is useful when using
this {func_type_lbl} in bigger projects e.g. in an IDE
(Integrated Development Environment).
Here is an example doc string for a simple function
......@@ -563,11 +565,12 @@ def docstring_issues(block_dets, *, repeated_message=False):
#### `{func_name}` lacks a doc string
You should probably add a doc tring to `{func_name}`
You should probably add a doc string to `{func_name}`
"""))
elif problem == DOCSTRING_TOO_SHORT:
if first and not repeated_message:
summary_bits.append((layout(f"""\
#### Function doc string too brief?
The doc string for `{func_name}` seems a little
......@@ -580,6 +583,7 @@ def docstring_issues(block_dets, *, repeated_message=False):
))
else:
summary_bits.append(layout(f"""\
#### Function doc string too brief?
The doc string for `{func_name}` seems a little short.
......
......@@ -48,15 +48,6 @@ def _get_shamed_names_title(reserved_names, bad_names, dubious_names):
title = 'Possibly some un-pythonic names'
return title
def _get_shamed_names_comment(shamed_names):
multiple_shamed_names = len(shamed_names) > 1
if multiple_shamed_names:
shamed_names_listed = utils.get_nice_str_list(shamed_names, quoter='`')
shamed_names_comment = f"{shamed_names_listed} are un-pythonic."
else:
shamed_names_comment = f"`{shamed_names[0]}` is un-pythonic."
return shamed_names_comment
def get_standard_assigned_names(block_dets):
"""
Only get names where we expect standard pythonic naming. So not named tuple
......@@ -201,6 +192,7 @@ def unpythonic_name_check(block_dets, *, repeated_message=False):
`high_scores` (not `highScores` or `HighScores`)
""")
pascal = layout("""\
In Python class names and named tuples are expected to be in Pascal
Case (also known as upper camel case) rather than the usual snake
case. E.g. `collections.ChainMap`
......
......@@ -17,6 +17,7 @@ UNPACKING_COMMENT = (
""")
+
layout(f"""\
#### Un-pythonic :-(
location = (-37, 174, 'Auckland', 'Mt Albert')
......@@ -26,6 +27,7 @@ UNPACKING_COMMENT = (
suburb = location[3]
#### Pythonic :-)
lat, lon, city, suburb = location
""", is_code=True)
+
......@@ -86,7 +88,9 @@ GENERAL_COMPREHENSION_COMMENT = layout(f"""\
LIST_COMPREHENSION_COMMENT = (
layout("""\
#### Example List Comprehension:
""")
+
layout(f"""\
......@@ -140,7 +144,9 @@ LIST_COMPREHENSION_COMMENT = (
DICT_COMPREHENSION_COMMENT = (
layout("""\
#### Example Dictionary Comprehension:
""")
+
layout(f"""\
......@@ -196,7 +202,9 @@ DICT_COMPREHENSION_COMMENT = (
SET_COMPREHENSION_COMMENT = (
layout("""\
#### Example Set Comprehension
""")
+
layout(f"""\
......
......@@ -2,6 +2,7 @@ import logging
from textwrap import dedent
from .cli_extras import md2cli
from ..utils import layout_comment as layout
"""
Note - displays properly in the terminal but not necessarily in other output
......@@ -10,18 +11,15 @@ e.g. Eclipse console
from .. import conf
TERMINAL_WIDTH = 220
TERMINAL_WIDTH = 80
SHORT_LINE = '-' * 2 ## at 3 long it automatically becomes a long line (at least, it does in my bash terminal on Ubuntu)
LONG_LINE = '-' * 120
MDV_CODE_BOUNDARY = "```"
def get_message(message_dets, message_level):
message = dedent(message_dets.message[message_level])
if message_level == conf.EXTRA:
message = dedent(message_dets.message[conf.MAIN]) + message
warning_str = 'WARNING:\n' if message_dets.warning else ''
message = dedent(warning_str + message)
message = dedent(message)
message = (message
.replace(f" {conf.PYTHON_CODE_START}", MDV_CODE_BOUNDARY)
.replace(f"\n {conf.PYTHON_CODE_END}", MDV_CODE_BOUNDARY)
......@@ -36,17 +34,22 @@ def display(snippet, messages_dets, *,
"""
logging.debug(f"{__name__} doesn't use in_notebook setting {in_notebook}")
md2cli.term_columns = TERMINAL_WIDTH
text = [md2cli.main(f"{LONG_LINE}\n"
"# SuperHELP - Help for Humans!\n"),
md2cli.main(f"\n{SHORT_LINE}\n"),
"Help is provided for your overall snippet "
"and for each line as appropriate.\n",
f"Currently showing {message_level} content as requested",
conf.MISSING_ADVICE_MESSAGE,
text = [
md2cli.main(layout(f"""\
# SuperHELP - Help for Humans!
Help is provided for your overall snippet and for each block of code
as appropriate.
Currently showing {message_level} content as requested.
{conf.MISSING_ADVICE_MESSAGE}
"""
)),
]
text.append(md2cli.main(dedent(
"## Overall Snippet\n"
f"{MDV_CODE_BOUNDARY}\n"
"## Overall Snippet"
f"\n{MDV_CODE_BOUNDARY}\n"
+ snippet
+ f"\n{MDV_CODE_BOUNDARY}")))
overall_messages_dets, block_messages_dets = messages_dets
......@@ -64,17 +67,19 @@ def display(snippet, messages_dets, *,
new_block = (line_no != prev_line_no)
if new_block:
block_has_warning_header = False
text.append(md2cli.main(
f'{LONG_LINE}\n## Code block starting line {line_no:,}'))
text.append(md2cli.main(dedent(
f"{MDV_CODE_BOUNDARY}\n"
f'## Code block starting line {line_no:,}'
f"\n{MDV_CODE_BOUNDARY}\n"
+ message_dets.code_str
+ f"\n{MDV_CODE_BOUNDARY}")))
prev_line_no = line_no
if message_dets.warning and not block_has_warning_header:
text.append("\n### Questions / Warnings")
text.append("\nThere may be some issues with this code "
"block you want to address.")
text.append(md2cli.main(layout("""\
### Questions / Warnings
There may be some issues with this code block you want to
address.
""")))
block_has_warning_header = True
## process message
message = get_message(message_dets, message_level)
......
......@@ -159,7 +159,7 @@ class AnsiPrinter(Treeprocessor):
return
is_text = (
el.text
bool(el.text)
or el.tag == 'p'
or el.tag == 'li'
or el.tag.startswith('h')
......@@ -187,7 +187,10 @@ class AnsiPrinter(Treeprocessor):
text = unescape(text)
else:
text = el.text
if cli_conf.BADLY_PARSED_UNDERSCORE in text:
text = text.replace(cli_conf.BADLY_PARSED_UNDERSCORE, '_') ## so __doc__ is displayed not 9595doc9595 when the source md has \_\_doc|_|_
text = text.strip()
admon_res = AnsiPrinter.handle_admonitions(text)
text, prefix, body_prefix, admon_lbl_used = admon_res
# set the parent, e.g. nrs in ols:
......
......@@ -105,4 +105,10 @@ BOUNDS2COLOUR = {
EMPH_BOUNDS: H3_COLOUR,
}
## https://en.wikipedia.org/wiki/Control_character
STX = '\x02' ## Start of transmission of non-data characters
ETX = '\x03' ## End of transmission
BADLY_PARSED_UNDERSCORE = f'{STX}95{ETX}'
LINK_START_ORD = ord("①")
......@@ -8,6 +8,118 @@ from pygments.lexers import get_lexer_by_name # @UnresolvedImport
from . import cli_colour, cli_conf
## monkey patch so invisible non-text is included in wrapping calculations making a mess of it
def _wrap_text_chunks_only(self, chunks):
"""_wrap_chunks(chunks : [string]) -> [string]
Wrap a sequence of text chunks and return a list of lines of
length 'self.width' or less. (If 'break_long_words' is false,
some lines may be longer than this.) Chunks correspond roughly
to words and the whitespace between them: each chunk is
indivisible (modulo 'break_long_words'), but a line break can
come between any two chunks. Chunks should not have internal
whitespace; ie. a chunk is either all whitespace or a "word".
Whitespace chunks will be removed from the beginning and end of
lines, but apart from that whitespace is preserved.
"""
lines = []
if self.width <= 0:
raise ValueError("invalid width %r (must be > 0)" % self.width)
if self.max_lines is not None:
if self.max_lines > 1:
indent = self.subsequent_indent
else:
indent = self.initial_indent
if len(indent) + len(self.placeholder.lstrip()) > self.width:
raise ValueError("placeholder too large for max width")
# Arrange in reverse order so items can be efficiently popped
# from a stack of chucks.
chunks.reverse()
while chunks:
# Start the list of chunks that will make up the current line.
# cur_len is just the length of all the chunks in cur_line.
cur_line = []
cur_len = 0
# Figure out which static string will prefix this line.
if lines:
indent = self.subsequent_indent
else:
indent = self.initial_indent
# Maximum width for this line.
width = self.width - len(indent)
# First chunk on line is whitespace -- drop it, unless this
# is the very beginning of the text (ie. no lines started yet).
if self.drop_whitespace and chunks[-1].strip() == '' and lines:
del chunks[-1]
while chunks:
## GPS - ignoring non-text
last_chunk = chunks[-1]
ignore_len = (
last_chunk.startswith(cli_conf.STX)
and last_chunk.endswith(cli_conf.ETX))
l = 0 if ignore_len else len(last_chunk)
# Can at least squeeze this chunk onto the current line.
if cur_len + l <= width:
cur_line.append(chunks.pop())
cur_len += l
# Nope, this line is full.
else:
break
# The current line is full, and the next chunk is too big to
# fit on *any* line (not just this one).
if chunks and len(chunks[-1]) > width:
self._handle_long_word(chunks, cur_line, cur_len, width)
cur_len = sum(map(len, cur_line))
# If the last chunk on this line is all whitespace, drop it.
if self.drop_whitespace and cur_line and cur_line[-1].strip() == '':
cur_len -= len(cur_line[-1])
del cur_line[-1]
if cur_line:
if (self.max_lines is None or
len(lines) + 1 < self.max_lines or
(not chunks or
self.drop_whitespace and
len(chunks) == 1 and
not chunks[0].strip()) and cur_len <= width):
# Convert current line back to a string and store it in
# list of all lines (return value).
lines.append(indent + ''.join(cur_line))
else:
while cur_line:
if (cur_line[-1].strip() and
cur_len + len(self.placeholder) <= width):
cur_line.append(self.placeholder)
lines.append(indent + ''.join(cur_line))
break
cur_len -= len(cur_line[-1])
del cur_line[-1]
else:
if lines:
prev_line = lines[-1].rstrip()
if (len(prev_line) + len(self.placeholder) <=
self.width):
lines[-1] = prev_line + self.placeholder
break
lines.append(indent + self.placeholder.lstrip())
break
return lines
textwrap.TextWrapper._wrap_chunks = _wrap_text_chunks_only
ansi_escape = re.compile(r"\x1b[^m]*m")
def get_code_hl_tokens():
......@@ -101,13 +213,16 @@ def rewrap(el, text, indent, prefix, terminal_width):
cols = max(terminal_width - len(indent + prefix), 5)
if el.tag == 'code' or len(text) <= cols:
return text
# this is a code replacement marker of markdown.py. Don'text split the
# this is a code replacement marker of markdown.py. Don't text split the
# replacement marker:
if text.startswith('\x02') and text.endswith('\x03'):
if text.startswith(cli_conf.STX) and text.endswith(cli_conf.ETX):
return text
dedented = textwrap.dedent(text).strip()
ret = textwrap.fill(dedented, width=cols)
return ret
new_lines = []
for line in text.split('\n'):
dedented = textwrap.dedent(line)
new_lines.extend(textwrap.wrap(dedented, width=cols)) ## had to monkey patch textwrap to prevent code replacement markers messing up line wrapping
new_text = '\n'.join(new_lines)
return new_text
def clean_ansi(s):
"""
......
......@@ -20,20 +20,21 @@ def get_ansi(md):
MD.convert(md)
ansi = MD.ansi + '\n'
ansi = cli_utils.set_hr_widths(ansi) + "\n"
# The RAW html within source, incl. fenced code blocks:
# phs are numbered like this in the md, we replace back:
PH = markdown.util.HTML_PLACEHOLDER
## The raw HTML within the source includes fenced code blocks.
## Placeholders are numbered like this in the md, we replace back
ansi_placeholder_tpl = markdown.util.HTML_PLACEHOLDER
stash = MD.htmlStash
for nr, ph in enumerate(stash.rawHtmlBlocks):
raw = unescape(ph)
if raw[:3].lower() == '<br':
for i, html_block_str in enumerate(stash.rawHtmlBlocks):
ansi_placeholder = ansi_placeholder_tpl % i
raw = unescape(html_block_str)
if raw.lower().startswith('<br'):
raw = '\n'
pre = '<pre><code'
if raw.startswith(pre):
_, raw = raw.split(pre, 1)
raw = raw.split('>', 1)[1].rsplit('</code>', 1)[0]
raw = tag_formatting.code(raw.strip(), from_fenced_block=True)
ansi = ansi.replace(PH % nr, raw)
ansi = ansi.replace(ansi_placeholder, raw)
return ansi
def main(md):
......
......@@ -6,7 +6,7 @@ def _get_vertical_padding_line(length):
def h(text, level):
level_colour = cli_conf.LEVEL2COLOUR.get(level)
bold = False
if level == 1:
if level <= 2:
text = text.center(cli_conf.TERMINAL_WIDTH)
vertical_padding_line = _get_vertical_padding_line(
length=cli_conf.TERMINAL_WIDTH)
......@@ -21,8 +21,8 @@ def h(text, level):
if level <= 2:
text = f"{vertical_padding_line}\n{text}\n{vertical_padding_line}"
bold = True
return '\n' + cli_colour.colourise(
text, level_colour, reverse=True, bold=bold)
return cli_colour.colourise(
text, level_colour, reverse=True, bold=bold) + '\n'
def h1(s, **_kwargs):
return h(s, level=1)
......@@ -40,7 +40,7 @@ def h5(s, **_kwargs):
return h(s, level=5)
def p(text, **_kwargs):
return cli_colour.colourise(text, cli_conf.T)
return cli_colour.colourise(text.strip('\n') + '\n', cli_conf.T)
def a(text, **_kwargs):
return cli_colour.colourise_low_vis(text)
......
import logging
import sys
from textwrap import dedent
from textwrap import dedent, wrap
from . import conf
......@@ -58,6 +58,24 @@ def layout_comment(raw_comment, *, is_code=False):
)
indented_lines = [f"{' ' * 4}{line}" for line in lines]
comment = f'\n'.join(indented_lines) + '\n' ## new line at end needed otherwise content of next str (if any) becomes part of code highlighting
comment = f'\n'.join(indented_lines) + '\n' ## new line at end needed otherwise content of next str (if any) becomes part of code highlighting
else:
comment = dedent(raw_comment)
raw_paragraphs = dedent(raw_comment).split('\n\n') ## we only split paragraphs if two new lines
new_paragraphs = []
for raw_paragraph in raw_paragraphs:
## replace internal new lines only - we need to preserve the outer ones
n_start_new_lines = (
len(raw_paragraph) - len(raw_paragraph.lstrip('\n')))
n_end_new_lines = (
len(raw_paragraph) - len(raw_paragraph.rstrip('\n')))
paragraph = raw_paragraph.strip()
one_line_paragraph = paragraph.replace('\n', ' ') ## actually continuations of same line so no need to put on separate lines
wrapped_paragraph_lines = wrap(one_line_paragraph)
new_paragraph = (
(n_start_new_lines * '\n\n')
+ '\n'.join(wrapped_paragraph_lines)
+ (n_end_new_lines * '\n\n')
)
new_paragraphs.append(new_paragraph)
comment = '\n' + '\n\n'.join(new_paragraphs)
return comment
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment