Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
2
2023-029
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
2023-029
2023-029
Commits
23439ff8
Commit
23439ff8
authored
Sep 05, 2023
by
Sumudu-Himasha-Ranaweera
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Fix: render the Unity WebGL build
parent
a07c0298
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
426 additions
and
72 deletions
+426
-72
Project/Backend/Server_Python/main.py
Project/Backend/Server_Python/main.py
+4
-0
Project/Frontend/SignConnectPlus/src/pages/video-to-sign-language/VideoTranslate/VideoTranslate.tsx
.../video-to-sign-language/VideoTranslate/VideoTranslate.tsx
+86
-72
Project/Frontend/SignConnectPlus/src/pages/video-to-sign-language/VideoTranslate/VideoTranslate.tsx.bkp
...eo-to-sign-language/VideoTranslate/VideoTranslate.tsx.bkp
+336
-0
No files found.
Project/Backend/Server_Python/main.py
View file @
23439ff8
...
@@ -54,6 +54,10 @@ origins = [
...
@@ -54,6 +54,10 @@ origins = [
"127.0.0.1:55553"
,
"127.0.0.1:55553"
,
"http://localhost:52823"
,
"http://localhost:52823"
,
"http://localhost:53826"
,
"http://localhost:53826"
,
"http://localhost:51373"
,
"http://localhost:51489"
,
"https://v6p9d9t4.ssl.hwcdn.net"
,
"https://64f66d39fdef493229b2ddd9--lambent-unicorn-97396a.netlify.app"
]
]
app
.
add_middleware
(
app
.
add_middleware
(
...
...
Project/Frontend/SignConnectPlus/src/pages/video-to-sign-language/VideoTranslate/VideoTranslate.tsx
View file @
23439ff8
...
@@ -24,7 +24,19 @@ import { CloudUploadOutlined, CopyOutlined, HighlightOutlined, TranslationOutlin
...
@@ -24,7 +24,19 @@ import { CloudUploadOutlined, CopyOutlined, HighlightOutlined, TranslationOutlin
import
axios
from
'
axios
'
;
import
axios
from
'
axios
'
;
import
{
MuiFileInput
}
from
'
mui-file-input
'
;
import
{
MuiFileInput
}
from
'
mui-file-input
'
;
import
{
useSnackbar
}
from
'
notistack
'
;
import
{
useSnackbar
}
from
'
notistack
'
;
import
{
useState
}
from
'
react
'
;
import
React
,
{
useState
}
from
'
react
'
;
// import Unity, { UnityContext } from "react-unity-webgl";
// ---------- * Unity Application * ------------------------------
// const unityContext = new UnityContext ({
// loaderUrl: "build/myunityapp.loader.js",
// dataUrl: "build/myunityapp.data",
// frameworkUrl: "build/myunityapp.framework.js",
// codeUrl: "build/myunityapp.wasm",
// });
// ==============================|| List ||============================== //
// ==============================|| List ||============================== //
...
@@ -35,6 +47,7 @@ const VideoTranslate = () => {
...
@@ -35,6 +47,7 @@ const VideoTranslate = () => {
const
[
value
,
setValue
]
=
useState
(
''
);
const
[
value
,
setValue
]
=
useState
(
''
);
const
[
translatedTextSi
,
setTranslatedTextSi
]
=
useState
(
''
);
const
[
translatedTextSi
,
setTranslatedTextSi
]
=
useState
(
''
);
const
[
translatedTextEn
,
setTranslatedTextEn
]
=
useState
(
''
);
const
[
translatedTextEn
,
setTranslatedTextEn
]
=
useState
(
''
);
const
[
showUnityWebGL
,
setShowUnityWebGL
]
=
useState
(
false
);
const
handleDropSingleFile
=
(
files
:
any
)
=>
{
const
handleDropSingleFile
=
(
files
:
any
)
=>
{
if
(
files
)
{
if
(
files
)
{
...
@@ -103,6 +116,10 @@ const VideoTranslate = () => {
...
@@ -103,6 +116,10 @@ const VideoTranslate = () => {
});
});
setTranslatedTextEn
(
response
.
data
.
translated_text_en
)
setTranslatedTextEn
(
response
.
data
.
translated_text_en
)
setTranslatedTextSi
(
response
.
data
.
translated_text_si
)
setTranslatedTextSi
(
response
.
data
.
translated_text_si
)
// Show the Unity WebGL build
setShowUnityWebGL
(
true
);
setLoading
(
false
)
setLoading
(
false
)
}
catch
(
error
)
{
}
catch
(
error
)
{
...
@@ -217,7 +234,7 @@ const VideoTranslate = () => {
...
@@ -217,7 +234,7 @@ const VideoTranslate = () => {
</
Card
>
</
Card
>
</
Grid
>
</
Grid
>
<
Grid
item
xs=
{
12
}
md=
{
6
}
>
<
Grid
item
xs=
{
12
}
md=
{
6
}
>
<
Card
sx=
{
{
p
:
5
,
minHeight
:
300
,
marginBottom
:
'
10px
'
,
marginRight
:
'
10px
'
}
}
>
<
Card
sx=
{
{
p
:
7
,
minHeight
:
300
,
marginBottom
:
'
10px
'
,
marginRight
:
'
10px
'
}
}
>
<
Box
display=
"grid"
gap=
{
5
}
>
<
Box
display=
"grid"
gap=
{
5
}
>
<
Stack
spacing=
{
2
}
>
<
Stack
spacing=
{
2
}
>
<
Grid
container
spacing=
{
1
}
>
<
Grid
container
spacing=
{
1
}
>
...
@@ -248,6 +265,67 @@ const VideoTranslate = () => {
...
@@ -248,6 +265,67 @@ const VideoTranslate = () => {
>
>
Translate
Translate
</
Button
>
</
Button
>
{
/* ... other JSX ... */
}
{
/* Conditionally render the Unity WebGL build */
}
{
showUnityWebGL
&&
(
<
iframe
src=
"https://64f66d39fdef493229b2ddd9--lambent-unicorn-97396a.netlify.app/"
width=
"700px"
height=
"700px"
// Adjust the height as needed
title=
"Unity WebGL"
style=
{
{
border
:
'
none
'
,
overflow
:
'
hidden
'
}
}
></
iframe
>
)
}
{
/* ... other JSX ... */
}
{
/* -------- Translated Sinhala Unicode ------------------------- */
}
<
Typography
variant=
"overline"
sx=
{
{
color
:
'
text.secondary
'
,
fontStyle
:
'
italic
'
,
marginBottom
:
2
}
}
>
Sinhala Unicode
</
Typography
>
<
TextField
sx=
{
{
marginBottom
:
2
}
}
fullWidth
value=
{
translatedTextSi
}
onChange=
{
handleChange
}
InputProps=
{
{
endAdornment
:
(
<
InputAdornment
position=
"end"
>
<
IconButton
onClick=
{
()
=>
onCopy
(
value
)
}
>
<
CopyOutlined
/>
</
IconButton
>
</
InputAdornment
>
)
}
}
/>
{
/* -------- Translated English Unicode ------------------------- */
}
<
Typography
variant=
"overline"
sx=
{
{
color
:
'
text.secondary
'
,
fontStyle
:
'
italic
'
}
}
>
English Unicode
</
Typography
>
<
TextField
fullWidth
value=
{
translatedTextEn
}
onChange=
{
handleChange
}
InputProps=
{
{
endAdornment
:
(
<
InputAdornment
position=
"end"
>
<
IconButton
onClick=
{
()
=>
onCopy
(
value
)
}
>
<
CopyOutlined
/>
</
IconButton
>
</
InputAdornment
>
)
}
}
/>
{
/* ----------------------------- */
}
</
Grid
>
</
Grid
>
</
Grid
>
</
Grid
>
{
loading
?
(
{
loading
?
(
...
@@ -264,20 +342,8 @@ const VideoTranslate = () => {
...
@@ -264,20 +342,8 @@ const VideoTranslate = () => {
)
:
(
)
:
(
<
div
>
<
div
>
{
/* -------- Translated Avatar ------------------------- */
}
<
Typography
variant=
"overline"
sx=
{
{
color
:
'
text.secondary
'
,
marginBottom
:
2
}
}
>
Translated Avatar
</
Typography
>
<
Paper
elevation=
{
3
}
sx=
{
{
p
:
2
,
maxWidth
:
600
,
margin
:
'
0 auto
'
,
marginBottom
:
3
}
}
>
<
video
controls
width=
"100%"
height=
"auto"
>
{
/* <source src="your-video-url.mp4" type="video/mp4" /> */
}
Your browser does not support the video tag.
</
video
>
</
Paper
>
{
/* -------- Translated Sinhala Unicode ------------------------- */
}
{
/* -------- Translated Sinhala Unicode ------------------------- */
}
<
Typography
variant=
"overline"
sx=
{
{
color
:
'
text.secondary
'
,
fontStyle
:
'
italic
'
,
marginBottom
:
2
}
}
>
{
/*
<Typography variant="overline" sx={{ color: 'text.secondary', fontStyle: 'italic', marginBottom: 2 }}>
Sinhala Unicode
Sinhala Unicode
</Typography>
</Typography>
...
@@ -297,10 +363,10 @@ const VideoTranslate = () => {
...
@@ -297,10 +363,10 @@ const VideoTranslate = () => {
</InputAdornment>
</InputAdornment>
)
)
}}
}}
/>
/>
*/
}
{
/* -------- Translated English Unicode ------------------------- */
}
{
/* -------- Translated English Unicode ------------------------- */
}
<
Typography
variant=
"overline"
sx=
{
{
color
:
'
text.secondary
'
,
fontStyle
:
'
italic
'
}
}
>
{
/*
<Typography variant="overline" sx={{ color: 'text.secondary', fontStyle: 'italic' }}>
English Unicode
English Unicode
</Typography>
</Typography>
...
@@ -317,7 +383,7 @@ const VideoTranslate = () => {
...
@@ -317,7 +383,7 @@ const VideoTranslate = () => {
</InputAdornment>
</InputAdornment>
)
)
}}
}}
/>
/>
*/
}
</
div
>
</
div
>
)
}
)
}
</
Stack
>
</
Stack
>
...
@@ -334,55 +400,3 @@ const VideoTranslate = () => {
...
@@ -334,55 +400,3 @@ const VideoTranslate = () => {
};
};
export
default
VideoTranslate
;
export
default
VideoTranslate
;
\ No newline at end of file
// // project import
// import { useState } from 'react';
// import MainCard from 'components/MainCard';
// import ScrollX from 'components/ScrollX';
// // assets
// //types
// // ==============================|| List ||============================== //
// const VideoTranslate = () => {
// const [sinhalaTranslation, setSinhalaTranslation] = useState('');
// const [singlishTranslation, setSinglishTranslation] = useState('');
// const handleConvertClick = () => {
// // Perform the video translation logic here
// // You can use the values from `sinhalaTranslation` and `singlishTranslation`
// };
// return (
// <MainCard content={false}>
// <ScrollX>
// <h3> Video Translation here </h3>
// <div>
// <h4>Sinhala Unicode Translation</h4>
// <textarea
// value={sinhalaTranslation}
// onChange={(e) => setSinhalaTranslation(e.target.value)}
// />
// </div>
// <div>
// <h4>Singlish Translation</h4>
// <textarea
// value={singlishTranslation}
// onChange={(e) => setSinglishTranslation(e.target.value)}
// />
// </div>
// <div>
// <button onClick={handleConvertClick}>Convert Video</button>
// </div>
// </ScrollX>
// </MainCard>
// );
// };
// export default VideoTranslate;
Project/Frontend/SignConnectPlus/src/pages/video-to-sign-language/VideoTranslate/VideoTranslate.tsx.bkp
0 → 100644
View file @
23439ff8
// project import
import
{
Box
,
Button
,
ButtonGroup
,
Card
,
CardContent
,
CardHeader
,
Container
,
Grid
,
IconButton
,
InputAdornment
,
LinearProgress
,
Paper
,
// Slider,
Stack
,
TextField
,
Typography
}
from
'@mui/material'
;
import
MainCard
from
'components/MainCard'
;
import
ScrollX
from
'components/ScrollX'
;
import
{
CloudUploadOutlined
,
CopyOutlined
,
HighlightOutlined
,
TranslationOutlined
}
from
'@ant-design/icons'
;
import
axios
from
'axios'
;
import
{
MuiFileInput
}
from
'mui-file-input'
;
import
{
useSnackbar
}
from
'notistack'
;
import
{
useState
}
from
'react'
;
// ==============================|| List ||============================== //
const
VideoTranslate
=
()
=>
{
const
[
file
,
setFile
]
=
useState
<
File
|
string
|
null
>
(
null
);
const
[
videoUrl
,
setVideoUrl
]
=
useState
(
''
);
const
[
loading
,
setLoading
]
=
useState
(
false
);
const
[
value
,
setValue
]
=
useState
(
''
);
const
[
translatedTextSi
,
setTranslatedTextSi
]
=
useState
(
''
);
const
[
translatedTextEn
,
setTranslatedTextEn
]
=
useState
(
''
);
const
handleDropSingleFile
=
(
files
:
any
)
=>
{
if
(
files
)
{
setFile
(
Object
.
assign
(
files
,
{
preview
:
URL
.
createObjectURL
(
files
)
})
);
setVideoUrl
(
URL
.
createObjectURL
(
files
));
}
};
const
handleChange
=
(
event
:
React
.
ChangeEvent
<
HTMLTextAreaElement
>
)
=>
{
setValue
(
event
.
target
.
value
);
};
// ----------------- Video Upload ------------------------------------------------
// const TranslateVideoToSignLanguage = async () => {
// if (file) {
// setLoading(true);
// const formData = new FormData();
// //@ts-ignore
// formData.append('video', file, file.name);
// try {
// const response = await VideoToSignLanguageService.videoTranslation(formData);
// const { translated_text_si, translated_text_en } = response.data;
// setTranslatedTextSi(translated_text_si);
// setTranslatedTextEn(translated_text_en);
// if (response.status == 200) {
// console.log(response.data);
// // setValue(response.data.predictions);
// } else {
// enqueueSnackbar('Something went Wrong!', { variant: 'error' });
// }
// // setLoading(false);
// } catch (error) {
// console.log(error);
// setLoading(false);
// enqueueSnackbar('Something went Wrong!', { variant: 'error' });
// }
// } else {
// enqueueSnackbar('Please select a file.', { variant: 'warning' });
// }
// };
async
function
uploadVideo
()
{
setLoading
(
true
)
if
(
file
)
{
const
formData
=
new
FormData
();
formData
.
append
(
'file'
,
file
);
try
{
const
response
=
await
axios
.
post
(
'http://127.0.0.1:8000/rest_pyton/uploaded_video'
,
formData
,
{
headers
:
{
'Content-Type'
:
'multipart/form-data'
,
},
});
setTranslatedTextEn
(
response
.
data
.
translated_text_en
)
setTranslatedTextSi
(
response
.
data
.
translated_text_si
)
setLoading
(
false
)
}
catch
(
error
)
{
console
.
error
(
'Error:'
,
error
);
setLoading
(
false
)
}
}
else
{
console
.
error
(
'No file selected.'
);
setLoading
(
false
)
}
}
const
{
enqueueSnackbar
}
=
useSnackbar
();
const
onCopy
=
(
text
:
string
)
=>
{
if
(
text
)
{
navigator
.
clipboard
.
writeText
(
text
);
enqueueSnackbar
(
'Copied!'
,
{
variant
:
'success'
});
}
};
return
(
<
MainCard
content
=
{
false
}
>
<
ScrollX
>
{
/* Content Here */
}
<
Container
sx
=
{{
padding
:
3
,
bgcolor
:
'#625D5D'
,
color
:
'#fafafb'
,
borderRadius
:
6
,
// boxShadow: '0px 4px 6px rgba(0, 0, 0, 0.1)' // Subtle box shadow
}}
>
{
/* Double Button Here */
}
<
ButtonGroup
disableElevation
variant
=
"contained"
aria
-
label
=
"Customized buttons"
sx
=
{{
marginBottom
:
'20px'
,
backgroundColor
:
'#ff3c3c'
,
// Change background color
'& .MuiButton-root'
:
{
// Apply styles to individual buttons
color
:
'white'
,
// Text color
'&:hover'
:
{
backgroundColor
:
'#000000'
// Change color on hover
}
}
}}
>
<
Button
sx
=
{{
bgcolor
:
'#ff3c3c'
,
padding
:
'10px 50px'
,
fontSize
:
'1.05rem'
,
// Larger font size
'& .anticon'
:
{
fontSize
:
'1.2rem'
,
// Larger icon size
},
}}
// variant={checkTranalationTypeForUpload()}
startIcon
=
{
<
CloudUploadOutlined
/>
}
// onClick={() => {
// setIsUploadFile(true);
// }}
>
Upload
</
Button
>
<
Button
sx
=
{{
bgcolor
:
'#ff3c3c'
,
padding
:
'10px 50px'
,
fontSize
:
'1.05rem'
,
// Larger font size
'& .anticon'
:
{
fontSize
:
'1.2rem'
,
// Larger icon size
},
}}
// variant={checkTranalationTypeForRecord()}
startIcon
=
{
<
HighlightOutlined
/>
}
// onClick={() => {
// setIsUploadFile(false);
// }}
>
Text
</
Button
>
</
ButtonGroup
>
{
/* Video uploading */
}
<
Box
sx
=
{{
flexGrow
:
1
}}
>
<
Card
>
<
CardHeader
title
=
"Upload a video | Drag & Drop or Select File"
/>
<
Grid
container
spacing
=
{
2
}
>
<
Grid
item
xs
=
{
12
}
md
=
{
6
}
>
<
Card
sx
=
{{
marginBottom
:
'20px'
,
marginLeft
:
'10px'
,
padding
:
'35px 10px'
}}
>
<
CardContent
>
{
/* ! Important */
}
{
/* @ts-ignore */
}
<
MuiFileInput
value
=
{
file
}
onChange
=
{
handleDropSingleFile
}
inputProps
=
{{
accept
:
'video/*'
}}
/>
<
Paper
style
=
{{
padding
:
'20px'
,
marginTop
:
'15px'
}}
>
<
Typography
variant
=
"h5"
align
=
"center"
gutterBottom
>
Preview
</
Typography
>
<
div
style
=
{{
marginTop
:
'20px'
,
textAlign
:
'center'
}}
>
{
file
?
<
video
src
=
{
videoUrl
}
width
=
"400"
controls
/>
:
<
p
>
No
Video
Selected
...</
p
>
}
</
div
>
</
Paper
>
</
CardContent
>
</
Card
>
</
Grid
>
<
Grid
item
xs
=
{
12
}
md
=
{
6
}
>
<
Card
sx
=
{{
p
:
5
,
minHeight
:
300
,
marginBottom
:
'10px'
,
marginRight
:
'10px'
}}
>
<
Box
display
=
"grid"
gap
=
{
5
}
>
<
Stack
spacing
=
{
2
}
>
<
Grid
container
spacing
=
{
1
}
>
{
/* <Grid item xs={12} md={6}> */
}
{
/* <h3>Set Sign Speed </h3> */
}
{
/* <Slider
defaultValue={30}
getAriaValueText={valuetext}
valueLabelDisplay="auto"
step={10}
marks
min={10}
max={110}
/> */
}
{
/* <h4>Speed - {speed}</h4> */
}
{
/* </Grid> */
}
<
Grid
item
xs
=
{
12
}
md
=
{
6
}
container
direction
=
"row"
justifyContent
=
"flex-start"
alignItems
=
"center"
>
<
Button
variant
=
"contained"
style
=
{{
width
:
'200px'
,
height
:
'60px'
,
fontSize
:
'20px'
}}
sx
=
{{
mb
:
3
}}
disabled
=
{
loading
}
onClick
=
{
uploadVideo
}
endIcon
=
{
<
TranslationOutlined
/>
}
>
Translate
</
Button
>
</
Grid
>
</
Grid
>
{
loading
?
(
<
Card
>
<
CardContent
>
<
LinearProgress
/>
<
center
>
<
Typography
variant
=
"h5"
component
=
"div"
sx
=
{{
marginTop
:
2
}}
>
Loading
...
</
Typography
>
</
center
>
</
CardContent
>
</
Card
>
)
:
(
<
div
>
{
/* -------- Translated Avatar ------------------------- */
}
<
Typography
variant
=
"overline"
sx
=
{{
color
:
'text.secondary'
,
marginBottom
:
2
}}
>
Translated
Avatar
</
Typography
>
<
Paper
elevation
=
{
3
}
sx
=
{{
p
:
2
,
maxWidth
:
600
,
margin
:
'0 auto'
,
marginBottom
:
3
}}
>
<
video
controls
width
=
"100%"
height
=
"auto"
>
{
/* <source src="your-video-url.mp4" type="video/mp4" /> */
}
Your
browser
does
not
support
the
video
tag
.
</
video
>
</
Paper
>
{
/* -------- Translated Sinhala Unicode ------------------------- */
}
<
Typography
variant
=
"overline"
sx
=
{{
color
:
'text.secondary'
,
fontStyle
:
'italic'
,
marginBottom
:
2
}}
>
Sinhala
Unicode
</
Typography
>
<
TextField
sx
=
{{
marginBottom
:
2
}}
fullWidth
value
=
{
translatedTextSi
}
onChange
=
{
handleChange
}
InputProps
=
{{
endAdornment
:
(
<
InputAdornment
position
=
"end"
>
<
IconButton
onClick
=
{()
=>
onCopy
(
value
)}
>
<
CopyOutlined
/>
</
IconButton
>
</
InputAdornment
>
)
}}
/>
{
/* -------- Translated English Unicode ------------------------- */
}
<
Typography
variant
=
"overline"
sx
=
{{
color
:
'text.secondary'
,
fontStyle
:
'italic'
}}
>
English
Unicode
</
Typography
>
<
TextField
fullWidth
value
=
{
translatedTextEn
}
onChange
=
{
handleChange
}
InputProps
=
{{
endAdornment
:
(
<
InputAdornment
position
=
"end"
>
<
IconButton
onClick
=
{()
=>
onCopy
(
value
)}
>
<
CopyOutlined
/>
</
IconButton
>
</
InputAdornment
>
)
}}
/>
</
div
>
)}
</
Stack
>
</
Box
>
</
Card
>
</
Grid
>
</
Grid
>
</
Card
>
</
Box
>
</
Container
>
</
ScrollX
>
</
MainCard
>
);
};
export
default
VideoTranslate
;
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment