thank you notes

Monday, November 22, 2010

android framework components

1.OMX components are decoders and others
2.MIO components are source/sink and others

A module can have multiple interfaces.A graph node is a module.It can have control interface.
At init time based on the queries on the control interface the other interfaces are linked.
Interfaces even exchange pool allocaters.for eg a pool can be created at MIO layer,its fd passes to
OMX layer and its used at this layer for allocations

physical memory can be allocated at MIO and passes to frame source to fill in.

3.node --- parser
node --- codec ----- OMX
node --- sink/source --- MIO
each node will have cfg files

4.currently pvcore expects
MIO IN --> OMX encoder --> PV recorder
PV player ---> OMX decoder --> MIO out
common MIO out is libopencorehw.so it talks to sf

5.on 3530 how arm commn with dsp
using dsp-bridge
special compiler used to gen binaries one for dsp one for arm
put them under directories where each will be picked by dsp or arm

general display

1.linux
x client
xserver
usermode lib
kernel driver

2.framebuffer hardware
its a complete hardware,takes block of memory and converts to pixels

framebuffer layers
opengl feeds instructions to gpu layer
gpu layer combines this with overlay devices like cursor,camera preview
gpu layer
videomemory layer
it fills the back buffer ,front buffer
display layer
dumps the picture

3.if some gl operations cannot be done by hw then it should fallback using sw .

4.an activity can give a set of gl commands and output memory block,gpu will render into this
memory and ,cpu will copy this memory to video ram for display

an activity can give set of gl commands and mapped vram memory to gpu,gpu will render into this and it will be directly displayed

software rendering can happen in 1 way,where it does all gpu operation and renders into
mapped vram.lot of temporary buffers required.lot of computations as well.

conclusion /dev/fb0 can be both accelerated and non accelerated device.

/dev/fb0 software renderer
/dev/devmem device is needed for hardware rendering

this abstracts device memory and this memory is mapped to user space directly
/dev/fb0 has an additional interface for hardware rendering. it can be disabled or enabled.

frame memory is mapped to activity whether it is hw/sw rendering.in hw rendering enclosing operations are done in hw,otherwirse in sw

Sunday, November 21, 2010

android 3d accel

1./dev/pmem_gpu0
2./dev/hw3d
3./dev/hw3dc
4./dev/graphics/fb0

google tv in short

mythtv plugin to a webbrowser
box needs a cable tv and broadband input
the browser should be able to search on cable and on broadband
the cable should respond to http queries
browser should have to display web and cable content
in the context of tv,web operations should be possible
It falls back to software copybit which is slow as hell, because each line of an image must be copied by the CPU (with memcpy), not even talking about software scaling. That's why copybit and 2D hardware are so important.

android flingers

1.learning has a wavy pattern
upper crest starts with abstraction
lower crest is concrete

upper(starts with some names)---lower(processes,dlls)--upper(associations)----lower(interface functions)---upper(categories,channels)----lower(context,binary modules)

mediaserver-->omx-->codec

omx buffer is sent with input buffer and offset,and returns output buffer

audio
======
af-->libaudio.so
af-->libaudio.so (alsa)

video
=====
uses EGL as interface
sf-->libhgl.so
sf-->libagl.so

codec
=====
uses OMX as interface
mediaserver-->omxnnn.so

pixelflinger/libhgl for an activity makes all defined surfaces and passes to sf
sf layers and composes them and renders using fb0

libhgl has inputs and outputs,inputs are surface data and surfaces
output is composed buffer
if this output buffer memory can be fed to fb0 the its efficient
but if it cannot be then a memcpy is required before pushing to fb0

eglbuffertypes
native
pushbuffer
fb0

All these are parallel
cpu
gpu multiple layer composer
display renderer hw
codec hw

an activity indirectly calls the eglswapbuffer,this will send data to gpu to compose,till its
completed activity waits,then the composed buffer is given to fb0

copybit.so can export any /dev/nnn,it has to define interface copybit.h,its used for 2d blit

android code assumes gpu/display to be same hw device /dev/graphics/fb0, different ioctls for operations

maybe before writing to gpu,clipping and other operations are done on figures ,a
nd later one by is rendered

android surface flinger

1.libhgl.so and libagl.so are loaded into sf.
2.every activity can have one or more surfaces
3.these surfaces may be of different types
4.some of these surfaces will use hw accel some are software
5.before pushing to fb0 surfaces will be composited

6.a combination of libEGL.so and libGLESv2.so will route gl calls to libhgl or libagl.

7.can an activity have hw surface and sw surface parallelly.can these two surfaces be composited together into fb0.
8.maybe both libhgl and libagl will be loaded parallelly and activity will decide if its using libagl or libhgl.
libagl and libhgl calls have one to one correspondence,so partial use of libagl and libhgl is not possible.

8.hardware codec support can be given in android by 2 ways.within a framework or outside
1.within framework --- get the working codec driver in openmax,modify it
take a template code from framework node,change it to work like above
this will work with standard android player.openmax can be adapted to fit into android framework
2.implement a player,within the player code interface with sf and af
if gstreamer already has the codec integrated,use it with a new player engine.

9.in a accelerated codec scenario application pushes a frame desciptor to driver to hardware,descriptor has source buff addr and destination buff addr.on decoding the frame the
frame is pushed to destination buffer.from here application takes it and send to fb0.

10.video acceleration === hardware codec
display acceleration === hadware gl(compositing)
audio accel ===
audio mixing === hadware level