Linux runs on huskylens2
R. Hatami 2026-10-26 06:19:17 253 Views22 Replies hi guys i was wondering if you could help me I bought a huskylens2 i managed to telnet into it using its ip and login root and it runs linux i can see the cam as /dev/video3 and it runs ffmpeg i want to get the bits from the camera i found a io file that has all the gpio pins in so i cant get a clear image and feed to process but i may need some help i know some really cool stuff to do with cameras and circuits like a dac to get an analog wave and a opamp to amplify it. it runs python too i tried to install opencv but it failed i found a compiled branch of k230 fw ver 1.8.1 which has opencv in im not sure if you can copy the files from 1.8.1 to 1.4.1 i havnt tried it yet but if you can read the bits in then you can read the bytes and make your own software for it with an interface if we find how that is displayed and your own applications
Hi guys now i have it all working i could do with some help and i wanted to build a community where we can work together. I know a lot of stuff.
Cameras are built to spec to what the human eye can see but it lets in the whole light spectrum in if you look at the light spectrum you can even see radio waves now i need to get the bits from the camera there is a iomux.py file on the sdcard which it looks like with some modification it will print bits then you put these through a digital to analog dac and get the wave out which has rgb values you can amplify this or lift the wave higher and you have digital luminance and a opamp for digital zoom
Also i need to look at running processes see if we can make a build that doesnt run much and runs python i found a buildchain for it
R. Hatami ok here it is a new interface i need help with it though im making a pytogether.com group
https://github.com/Rzaandroid/Huskylens2
download the files place them in ~/
telnet in and run ./runmenu.sh
first icon starts xiao2term xiaozhi output to terminal to grep for text and launch apps
but the second icon needs to be launched to restart the menu executable to start all the services to listen for audio
then press the second icon
to run test python code beginning of a framework ive started in python and opencv to get multiple feeds keep a uninterrupted live feed and process the other feeds with separate threads
Please download the project and run it let me know what you think
Telegram is up http://t.me/+f2ibcqqII1gzY2E0
github is up https://github.com/Rzaandroid/Huskylens2
pytogether is up ZP6Y3uxSeWBN click join group and add this key
R. Hatami i need to make this autostart when huskylens2 starts
i have to search for .py in folder and add all py files and autogenerate sh files to kill processes then run python
i have to change the icon color when you click icon properly
i have a bug with xioa.sh where the outputs doesnt print to term but works if you launch the python file and have menu process running
ok ive made a menu now to run your own python applications
it has icons and you use the touchscreen to select what to run it works too
it launches a shell script to launch your python applications
you can add in the shell script which processes to kill
like menu or python3 if another application is running
i need to make a github to host all the files on and will do this tonight for tomorrow
R. Hatami i need to add something in there that checks how many .py files there are in a folder then adds an icon to launch the application
OK HOW TO RUN OPENCV AND YOUR OWN PYTHON CODE
killall menu
python3 ./camera_drm.py
make file camera_drm.py and past code in it
I use vi on there in telnet
[shift] I to edit insert
[esc] to escape edit insert
[:] [w] to write
[:] [q] to quit
code is here
https://pastebin.com/rednW63G
R. Hatami here is what ive done so far python has no binding to display an image i couldnt find a framebuffer either so i have some code to write to dri drm and it draws to the screen directly
i havnt yet implemented any processing i have a clean camera feed
i use netcat to send chars to the arduino wifi r4 board to control the motor driver and a script to handle pressing the key and release so it stops
i have xioazhi working printing the text to the console now i can search for keywords and execute python i had an error with permissions so i need to fix this
I have a script that kills the main running app menu runs my python code starts running python code to grame the camera feed and dispaly it on the screen
starts xioazhi output from shm shared memory with all the text
then runs menu again to start all the services it needs to run
here is a video i was super tired last night i finished late in the morning but i got it all working
http://filebin.net/2ihm3xe1a8xeuxn2
R. Hatami I like to work on things together with people in a community so please msg me on discord reza2336 and we can make a group i also made a telegram group which would be cool because the robot could post to it its log.
try telegram
http://t.me/Huskylens2
let me know if it works or not
R. Hatami the telegram link is wrong here it is
https://t.me/+f2ibcqqII1gzY2E0

ok so my robot now drives around i use a 13 pounds cheap plastic arduino bot
it looks for a person then looks for a face
in the future i want to ask who am i looking at and it will put the text in of the name and store the image to detect later which is machine learning
then i want to ask what are you doing then you say what you are doing.
when it docks it calls a friend on dfrobot community with huskylens and tells it my owner [reza] [is working on the computer]
it can keep a log aswell and read it to you if you go out and come back
example
at 12:00 i went to the kichen
at 12:05 i found a fly
at 12:06 i went to the lounge
at 12:08 my battery status showed it low
at 12:08 i went to the induction charging point to charge my battery i called a friend and talked to its owner here is the recording i asked what robot do you have, do you have a message for my owner, my owner implemented these new methods in my code etc.
at 12:30 my owner [reza] came home and asked can i see an image from your robot and it takes an image from the friends robot and sends it
if you use fall detection your robot could call all its friends for help and send the image and address
R. Hatami it can post it to your website too
TUTORIAL HOW TO DO COMPUTER VISION AND MACHINE LEARNING THE BEST WITHOUT WAITING EACH TIME
PART2
ok this is how its done
while True:
ret, frame = cam.read()
frame2 = frame
frame3 = frame
if ret:
if takephoto == 1:
takephoto=0
#take()
t1 = threading.Thread(target=edgedetection(), args=(4,))
t2 = threading.Thread(target=persondetection(), args=(4,))
t3 = threading.Thread(target=facedetection(), args=(4,))
#t4 = threading.Thread(target=doordetection(), args=(4,))
t1.start()
t1.join()
t2.start()
t2.join()
t3.start()
t3.join()
R. Hatami OK I MESSED UP THAT POST
TUTORIAL HOW TO DO COMPUTER VISION AND MACHINE LEARNING THE BEST WITHOUT WAITING EACH TIME
in here you have a stream of frames it is in a while loop
while True:
# Capture frame-by-frame
ret, frame = cap.read()
#add this
frame1 = frame
frame2 = frame
frame3 = frame
R. Hatami this make 3 video streams and the frame1 rename to live so this is your clear live feed
frame 2 is processed by a separate thread and is edges so make thread run make grey and edge detection rename to edges
frame3 is door detection on the edges so change frame3 to door = edges and make a new thread to process door and look for a door by matching a line and rotating it like hough
TUTORIAL HOW TO DO COMPUTER VISION AND MACHINE LEARNING THE BEST WITHOUT WAITING EACH TIME
in here you have a stream of frames it is in a while loop
while True:
# Capture frame-by-frame
ret, frame = cap.read()
R. Hatami ok add
while True:
# Capture frame-by-frame
ret, frame = cap.read()
frame1 = frame under
i still want to get the bits out using something like this or /dev /bin/k230_iomux.py
so i can see other parts of the light spectrum not just visible anyone know electronics and dacs
R. Hatami ok i wrote a python file and found opencv2 is installed i used this i commented the imshow and waitkey as its not implemented im not sure how to display the image im a noob at python but not opencv i dont know if you can write the image to the frame buffer or something to display it so you dont need to compile or add intergrations i can try it tomorrow
import numpy as np
import cv2 as cv
cap = cv.VideoCapture(3)
if not cap.isOpened():
print("Cannot open camera")
exit()
while True:
# Capture frame-by-frame
ret, frame = cap.read()
# if frame is read correctly ret is True
if not ret:
print("Can't receive frame (stream end?). Exiting ...")
break
# Our operations on the frame come here
gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
# Display the resulting frame
#cv.imshow('frame', gray)
#if cv.waitKey(1) == ord('q')
# When everything done, release the capture
cap.release()
#cv.destroyAllWindows()
R. Hatami to use the cam you must after telnetting in
killall /opt/menu
then run this to capture one frame
ffmpeg -f v4l2 -video_size 1280x720 -i /dev/video3 -frames:v 1 photo.jpg
or write a python file .py
with this to capture 3 frames from the camera
import ffmpeg
(
ffmpeg.input("/dev/video3")
.output('frame_%d.png', vframes=3)
.run()
)
now i have a frame i can make a stream and copy the frame to make multiple streams to process then used separate threads to process each stream keeping the live feed on main thread unprocessed
R. Hatami Huskylens2 Runs Linux
Get IP: [System settings] [WiFi] shows the ip its connected to
login: root
telnet [husky ip]
[top] shows processes running
kill pid
of /opt/menu
and the other pid
then run /opt/menu
shows what starts up
i found /dev/video3
the device also runs pyhton on it
cat /dev
shows there is a gpio chip
gpiodetect
gpioset
gpioget
dmesg
cat /etc/*release
from buildroot.org
pip install opencv-python
pip install numpy
ls /dev/i2c*
cat /opt/detect_sensor.py
R. Hatami 
