Fatal error after installing PuPHPeteer (node issue)

sage9

#1

Hi,
After installing PuPHPeteer,
https://github.com/nesk/puphpeteer

i get this Fatal error when i test a basic function:

Fatal error: Uncaught Symfony\Component\Process\Exception\ProcessFailedException: The command “‘node’ ‘–inspect’ ‘/app/public/app/themes/musicdata/vendor/nesk/rialto/src/node-process/serve.js’ ‘/app/public/app/themes/musicdata/vendor/nesk/puphpeteer/src/PuppeteerConnectionDelegate.js’ ‘{“idle_timeout”:60,“log_node_console”:false,“log_browser_console”:false}’” failed. Exit Code: 127(Command not found) Working directory: /app/public Output: ================ Error Output: ================ sh: 1: exec: node: not found in /app/public/app/themes/musicdata/vendor/nesk/rialto/src/ProcessSupervisor.php on line 309

Here is my php script:

namespace App;

use Roots\Sage\Config;
use Roots\Sage\Container;
use Nesk\Puphpeteer\Puppeteer;
use Nesk\Rialto\Data\JsFunction;
use Nesk\Rialto\Exceptions\Node;
use Nesk\Puphpeteer\Resources\ElementHandle;
use Goutte\Client;
use Symfony\Component\DomCrawler\Crawler;
use DOMDocument;
use DOMXPath;

/**
* Scrapping with Google Puppeteer (php wrapper version)
* @link https://github.com/nesk/puphpeteer
* @link https://dev.to/alanmbarr/scraping-html-with-php-node-and-puppeteer-10m2
*/
function md_extract_with_puppeteer( $url ){

	$url = 'https://alanmbarr.github.io/HackMidWestTimeline/';

	if(empty($url)) return;
	
	$puppeteer = new Puppeteer([
    // Node's executable path
    'executable_path' => 'node',

    // How much time (in seconds) the process can stay inactive before being killed (set to null to disable)
    'idle_timeout' => 60,

    // How much time (in seconds) an instruction can take to return a value (set to null to disable)
    'read_timeout' => 30,

    // How much time (in seconds) the process can take to shutdown properly before being killed
    'stop_timeout' => 3,

    // A logger instance for debugging (must implement \Psr\Log\LoggerInterface)
    'logger' => null,

    // Logs the output of console methods (console.log, console.debug, console.table, etc...) to the PHP logger
    'log_node_console' => false,

    // Enables debugging mode:
    //   - adds the --inspect flag to Node's command
    //   - appends stack traces to Node exception messages
    'debug' => true,
	]);

	/*
	$browser = $puppeteer->launch([
    'args' => ['--no-sandbox', '--disable-setuid-sandbox'],
	]);
	*/

	$browser = $puppeteer->launch();

	$page = $browser->newPage();

	$page->goto($url);

	$data = $page->evaluate(JsFunction::createWithBody('return document.documentElement.outerHTML'));
	
	var_dump($page->content()); // Prints the HTML

	$browser->close();

}

the error appears after this line:
$puppeteer = new Puppeteer

I test this on my local machine,i use Local By Flywheel (i’ve got the same error on my distant apache server).
My config is:
Web Server: Apache
PHP Version: 7.1.4
MySql: 5.5.60
Wordpress Version: 4.9.8

Node is installed globally, node version: 8.6.0

Here is my composer.json:

{
  "name": "roots/sage",
  "type": "wordpress-theme",
  "license": "MIT",
  "description": "WordPress starter theme with a modern development workflow",
  "homepage": "https://roots.io/sage/",
  "authors": [
    {
      "name": "Ben Word",
      "email": "ben@benword.com",
      "homepage": "https://github.com/retlehs"
    },
    {
      "name": "Scott Walkinshaw",
      "email": "scott.walkinshaw@gmail.com",
      "homepage": "https://github.com/swalkinshaw"
    },
    {
      "name": "QWp6t",
      "email": "hi@qwp6t.me",
      "homepage": "https://github.com/qwp6t"
    }
  ],
  "keywords": ["wordpress"],
  "support": {
    "issues": "https://github.com/roots/sage/issues",
    "forum": "https://discourse.roots.io/"
  },
  "autoload": {
    "psr-4": {
      "App\\": "app/"
    }
  },
  "require": {
    "php": ">=7.1",
    "composer/installers": "~1.0",
    "illuminate/support": "~5.6",
    "roots/sage-lib": "~9.0.1",
    "soberwp/controller": "~9.0.0-beta.4",
    "jwilsson/spotify-web-api-php": "^2.4",
    "jolitagrazyte/discogs-api": "dev-master",
    "fabpot/goutte": "^3.2",
    "log1x/socialproof": "^0.1.2",
    "nesk/puphpeteer": "^1.4"
  },
  "require-dev": {
    "squizlabs/php_codesniffer": "^2.8.0",
    "roots/sage-installer": "~1.3"
  },
  "scripts": {
    "test": ["phpcs"],
    "post-create-project-cmd": [
      "Roots\\Sage\\Installer\\ComposerScript::postCreateProject"
    ]
  }
}

The issue comes from Puppeteer. i have installed it in my sage theme using the following commands:

composer require nesk/puphpeteer
npm install @nesk/puphpeteer

I think the package can’t execute some node commands, because i get a Code 27 (sh: 1: exec: node: not found).

My purpose is to analyse some javascript rendered web pages.
Any help would be appreciate, i’m really stuck now :face_with_raised_eyebrow:


#2

Hey @mecanographik

Have you confirmed that the command node runs inside of the Local by Flywheel VM where your site is running, not just on your host system? In other words, if you SSH into the Local VM and run node -v, does it work? I believe you would need it installed there, and it’s not installed out of the box.


#3

Hi @mmirus
Thanks for your answer. I have SSH into the local VM (inside the current website by right click and the site name in the Local By Flywheel interface, is that what you mean?) and i ran node -v and it didn’t work, i get
node: command not found

So do i have to do this?

I have tried this first command:

# Install sudo, curl, git and apt-transport-https packages.
apt-get install sudo curl git apt-transport-https

but it didn’t work, i get this response:

    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    Package curl is not available, but is referred to by another package.
    This may mean that the package is missing, has been obsoleted, or
    is only available from another source
    E: Unable to locate package sudo
    E: Package 'curl' has no installation candidate
    E: Unable to locate package git
    E: Unable to locate package apt-transport-https

I’m not sure i’m on the right track :thinking:

How would you process? And why do i have the same issue on my Stage distant server ?


#4

@mmirus

I have to install Node in SSH on My VM with the following commands:

# update APT repositories before installing anything else
sudo apt-get update

apt-get install xz-utils

cd ~
wget https://nodejs.org/dist/v8.6.0/node-v8.6.0-linux-x64.tar.xz

tar xvf node-v8.6.0-linux-x64.tar.xz

#rename the extracted folder to the more convenient node name
mv node-v8.6.0-linux-x64 node
 
# install g++ to compile stuff
sudo apt-get install -y g++

apt-get install -y nodejs

apt-get install nodejs-legacy

#edit bash profile with nano, add this line:
alias node=nodejs 

Now when i enter :
node -v

It says:
v0.10.29

And now, i have a different fatal error ! when i navigate to my local website (code:8):

ErrorException: The command "'node' '/app/public/app/themes/musicdata/vendor/nesk/rialto/src/node-process/serve.js' '/app/public/app/themes/musicdata/vendor/nesk/puphpeteer/src/PuppeteerConnectionDelegate.js' '{"idle_timeout":60,"log_node_console":false,"log_browser_console":false}'" failed. Exit Code: 8(Unknown error) Working directory: /app/public Output: ================ Error Output: ================ /app/public/app/themes/musicdata/vendor/nesk/rialto/src/node-process/serve.js:3 const ConsoleInterceptor = require('./NodeInterceptors/ConsoleInterceptor'), ^^^^^ SyntaxError: Use of const in strict mode. at Module._compile (module.js:439:25) at Object.Module._extensions..js (module.js:474:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Function.Module.runMain (module.js:497:10) at startup (node.js:119:16) at node.js:906:3 (View: /app/public/app/themes/musicdata/resources/views/partials/content-page.blade.php) (View: /app/public/app/themes/musicdata/resources/views/partials/content-page.blade.php) in /app/public/app/themes/musicdata/vendor/nesk/rialto/src/ProcessSupervisor.php on line *309*

any idea?


#5

Hey @mecanographik - unfortunately this may be a little far away from Sage to troubleshoot in-depth here. Your best bet might be to go directly to one of the source projects for support (e.g., Local for help setting up that environment or Puphpeteer for help running it).

That said, two quick suggestions:

First, the version of Node you ended up with (0.10.29) is super, super old (and the instructions were a bit weird). I would remove it and install a newer version following the below instructions. I tested them in a Local VM, so they should work for you.

apt-get update
apt-get install curl ca-certificates
curl -sL https://deb.nodesource.com/setup_8.x | bash -
apt-get install -y nodejs
  • Local logs you in as root automatically, so you don’t need to use sudo for anything.
  • I found I had to run apt-get update twice, as the first time it gave me a ‘size of file is not what the server reported’ warning.
  • Without the ca-certificates, your curl request will fail (and it will fail silently because of the -s flag), which means the node repository won’t be added to your sources, which means you will be installing an old version of node instead of the 8.x version. You should see a bunch of messages while curl -sL https://deb.nodesource.com/setup_8.x | bash - is running.

When you’ve finished running all of those commands, node -v should output v8.12.0.

Second, instead of running Puphpeteer in your VM, it may be worth exploring running it on your host system. Folks take different approaches–some prefer running build and testing commands in the VM, some don’t. You may have better luck the other way.


#6

Hi, Thanks so much to take the time with my issue, i really appreciate.
First, i couldn’t make your command lines work, i had this error message:
E: The method driver /usr/lib/apt/methods/https could not be found.
N: Is the package apt-transport-https installed?
Error executing command, exiting

but i have managed to remove and reinstall my nodes package using the following instructions:

apt-get remove --purge nodejs
apt-get update
apt-get install nodejs

so now when i do node -v
i getv11.0.0

but when i do nodejs -v
i get v0.10.29 :flushed:

now when i test my function on the website i only get this fatal error:

Fatal error: Uncaught Nesk\Rialto\Exceptions\Node\FatalException: Chromium revision is not downloaded. Run "npm install" or "yarn install" in /app/public/app/themes/musicdata/vendor/nesk/rialto/src/ProcessSupervisor.php on line *307*

but it doesn’t change anything when i run yarn (or npm) install …

i will think about you last advices, but i think Puphpeteer should work on the host system, via Sage theme normally…